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Computational Complexity as a Scientific Metaphor 

Christos H. Papadimitnou University of California, San Diego, La Jolla, CA 92093 

Computational Complexity is a mathematical theory which strives to explain the large amounts of 
computational resources (typically, computation time) apparently required for the solution of many 
important computational problems. During the past twenty years, research in Complexity has focussed 
on problems of combinatorial nature, with occasional excursions into algebraic and numerical 
computation. One of the achievements of this theory has been the classification of computational 
problems according to their potential for an efficient solution. Of these classes, perhaps best known 
among researchers in other fields is that of iVP-complete problems, a class of problems currently believed 
to be intractable, requiring exponential amounts of computauon. 

In a literal application of these ideas, we are faced with a computational problem suspected to be 
difficult, and we use the tools of Complexity Theory to formally prove it intractable, that is, highly 
unlikely to be solvable by algonthms with acceptable performance This talk will describe a genre of 
scientific argument which can be viewed as a metaphoncal application of the ideas of Complexity 
Theory. We are faced with a difficult mathematical problem, not of a computational nature, and we use 
the intractability of a derived computational problem (sometimes not of direct practical interest) as 
evidence for the difficulty of the original (mathematical) problem We illustrate this point by examples 
from several areas, including Distributed Control, Game Theory, and Operations Research. 
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TECHNICAL SESSIONS 


Monday, 9 a.m. -12 m. 


SESSION MAI 
SPREAD SPECTRUM 


A Comparison of the Performance of Two Types of Narrowband Interference Rejection Techniques 
in DS-Spread Spectrum Systems 

Sophie Y Dayot and Laurence B Milstein Department of Electrical and Computer Engineering, San 
Diego State University, San Diego, CA 92182, and Department of Electrical and Computer Engineering, 
University of California, San Diego, La Jolla, CA 92093 

In this paper, a companson is made of the performance of two narrowband interference rejection 
schemes when used in conjuncuon with a direct sequence spread spectrum receiver. The first scheme is 
based upon the use of an esumation filter designed by solving the Wicncr-Hopf equauon, and the second 
technique makes use of a transform domain processing structure. Probabilities of bit error are presented 
for both systems under the conditions of cither tone or narrowband Gaussian interference, and it is shown 
that the former scheme is superior for tone interference. However, when the interference has a finite 
bandwidth, in most cases the latter technique yields better performance. 

Direct Sequence Spread Spectrum with Random Signature Sequences: A Large Deviations Analysis 
John S. Sadowsky and Randall K. Bahr School of Electrical Engineering, Purdue University, West 
Lafayette, IN 47907, and Dept, of Electrical & Computer Eng , University of Arizona, Tucson, AZ 85721 
This paper investigates the bit error probability of a direct sequence spread spectrum multiple access 
communications system using large deviations theory Let m denote the number of mterfcnng signals and 
n denote the length of the direct sequence signature sequence We consider the limit is as n s ~ with m 
fixed. The interfenng signals have random phases and delays with respect to the desired signal Let 9 
denote the vector of interfenng signal phases and delays, and define P b (9) = P (bit error ||9 = 0) A 
straightforward applicauon of modem large deviations theory indicates that Ps(0) vanishes exponentially 
as n —t ~ with rate / (0) > 0. Our main theoretical contnbution is to prove the limit 

P b =E[P b ( 9)) ~ C «' <3 '" tl) ' 2 cxp(-/ n) 

where / is the worst case exponential rate, that is, / = min 9 / (d), and - means that the ratio tends to 1 as 
A o An important biproduct of our analysis is that it leads to numcncally clficicnt means to esumate 
P b for finite n 


Exact Analysis of Asynchronous Frequency-Hop Spread-Spectrum Multiple-Access Networks 
Kyungwhoon Chcun and Wayne E Stark Department of Electrical Engineering and Computer Science, 
University of Michigan, Ann Arbor, Ml 48109 

In this paper, we derive the exact probability of error of an Asynchronous Frequency-Hop Spread 
Spectrum Multiple-Access (aFHSS-MA) Network transmitting one BFSK modulated bit per hop using 
the theory of spherically symmetric random vectors. It is assumed that Markov hopping patterns are 
employed. We show that the usual approximation that the probability of error is ''h (’/i-approximation) 
whenever a hop is hit by multiple-access interference, is excessively pessimistic and grossly 
underestimates the multiple-access capability of AFHSS-MA networks. Also, we show that using the '/4- 
approximation has led to misleading conclusions. For example, we show that contrary to popular belief, a 
system using perfect side-information to erase the hops that were hit performs much worse than systems 
making simple hard decisions without side-informaUon We show that the same techniques used to derive 
the probability of error can be used to denve the channel statistics of an AFHSS-MA channel when 



Viterbi ratio thresholding (VRT) is employed, and present numerical results to show that the VRT offers 
significant improvements m performance compared to simple hard decisions at a small increase in 
complexity. 


Quadratic Congruential Coding and its Implementation in Frequency-Hop Spread-Spectrum 
Communication Systems 

Zoran I. Kostic and Edward L. Titlebaum Department of Electrical Engineering, University of 
Rochester, Rochester, NY 14627 

The code-dependent performance of frequency-hop spread-spectrum multi-user communication 
systems based on linear congruence (LC) codes and a subset of Reed-Solomon Codes (Costas or 
Einarsson's codes) is presented for the case when Utree sources of interference are present: mutual 
interference between system users (between-users interference), interference between signals onginating 
from the same user due to the unknown signal propagation time and multiple transmissions (single-user 
signal interference), and doppler. Unsatisfactory performance in the case of single-user signal interference 
is used as a motivation for introducing a new system which uses quadratic congruence (QC) codes. The 
performance of this system is evaluated. It is close to the optimal in case of between-users interference, 
and the best known in case of single-user signal interference. Doppler interference performance is not 
affected by the code choice. The symmetry of QC codes is recalled and used for the design of truncated 
QC codes. Fullness of truncated QC codes eliminates the receiver design problems associated with non- 
lullness of QC codes. 


"CDMA-FDMA Hybrid Protocol" for Distributed Multihop Spread-Spectrum Packet Radio 
Networks 

Shingo Tanaka, Akihiro Kajiwara, and Masao Nakagawa Faculty of Science and Technology, Keto 
University, 3-14-1 Hiyoshi Kohohiku, Yokohamashi 223, Japan 

CDMA (code-division multiple access) packet radio network has the mem that it resists lading and 
jamming, but also has the dement of non-zero cross-coneiauon between different signals causing the 
"near far problem". To keep ment of CDMA and simultaneously have characteristic of anti-near far 
problem, we propose the "CDMA-FDMA (frequency-division multiple access) hybrid protocol” for 
distnbuted muluhop packet radio networks. 

In this protocol, spread frequency band is divided to some areas. In each divided band, CDMA is 
used The cross-correlation of spread spectrum signals in different band equals to zero Therefore, if we 
distnbute some (at most 4 or 5) frequency bands to some terminals placed close to each other, higher SIR 
(signal-to-intcrfercncc ratio) than CDMA-only protocol is expected. 

>From the operabon of dividing frequency band to some areas, the cross-correlation of SS signals in 
same band is higher than that of CDMA, because of the narrowc band width than that of CDMA. But the 
zero cross-correlation of FDMA has higher effect than the above problem, if the distribution of frequency 
bands is ideal. 

In four examples of networks, we explain the result that the hybrid protocol has higher SIR than 
CDMA only. 


Error Correcting Coding and Pseudo-Random Interleaving Scheme Against Intelligent Partial 
Time Jammers 

Philippe R. Sadot, Marc M. Darmon, and Sami Harari Alcatel Transmissions par Faisreaux Hertziens, 
Military Transmissions Department, 92301 Levallois-Perret, France, and Groupe d'Etude du Codage de 
Toulon, Universite de Toulon et du Var, 83130 La Garde, France 

The usual techniques of antijamming against partial time narrow band jammers consisung of 
spectrum spreading and error correcting coding are not efficient in the case of most military microwave 


3 


links because of the minimum data rate to transmit and of the channel bandwidth limitation. 

A new strategy of antijamming is proposed, which consists in admitting that all information is 
erased during the short pulses of the jammer, and in trying to reconstitute the lost mfoimation by means 
of an error correcting coding and interleaving scheme. 

In order to cope with intelligent jammers we have defined a new pseudorandom interleaving that 
features both characteristics of guaranteed minimum distance between symbols of the same codework and 
cryptographic complexity. It uses a high complexity non-linear pseudorandom generator, and a set of 
three rules that insure the minimum distance property. 

The coding scheme and the interleaving device will be described. 

The BER performance of the coding and interleaving scheme has been derived by means of 
algebraic techniques and the MACSYMA software, and will be explained. 


Finite Memory Recursive Solutions for the Equilibrium and Transient Analysis of G/M/l-Type 
Markov Processes with Application to Spread Spectrum Multiple Access Networks 
Ganmella Rama Murthy and Edward J Coyle School of Electrical Engineering, Purdue University, West 
Lafayette, IN 47907 

G/M/l-type Markov processes provide natural models for die activity on multiple access networks 
Efficient recursive solutions for the equilibnum and transient analysis of these processes are therefore of 
considerable interest. 

In this paper, a state space expansion entenon called level entrance direction information is 
introduced for G/M/l-type Markov processes It is then shown that this entenon leads to a new class of 
recursive solutions, called finite memory recursive solutions, for the equilibnum probabilities A finite 
memory recursive solution of order k has the form 

*.♦*“*^,+*,+^2+ • + W,W*. 

where ir„ is the vector of limiting probabilities of states on level n of the process and VV,, 1 < i arc 
nxn matrices. 

It is also shown that, utilizing the concept of LEDI completeness, a finite memory recursive solution 
for the Laplace transform of the vector of state occupancy probabilities at lime r can be found. Such a 
recursive solution has the form 

ff«**(i)=rr.(r)W,(s)+ir..i(s)W 2 (s)+ ■ +*„**.,(s)W*(sj 

Thus, this recursive solution provides a tractable method for the transient analysis of G/M/l-type Markov 
processes. The rclauonship cetwcen finite memory recursive soluuons and matnx geometne solutions is 
also explored 

A new G/M/l-type Maikov process model of Spread Spectrum Slotted ALOHA (SSSA) networks is 
developed, this model provides a finite memory recursive soluuon for the computation of the equilibnum 
and transient distribution of the number of busy users. It also allows many analytical results such as the 
necessary and sufficient condiuons for the stability and exact closed form expression for the expected 
delay to be found 


Moment Methods for Estimation of Fine Time Synchronization Error in FH/MFSK Systems 
L J. Mason and E. B. Felstead Communications Research Centre, Ottawa, Canada 

Time synchronization in frequency-hopped (FH) systems consists of two stages The coarse stage 
reduces the alignment error between the transmitter and the receiver hop periods to less than one half hop 
penod. The fine stage reduces this error to typically a few percent of a hop penod. Methods arc described 
which give low-biased, consistent estimates for the fine time synchronization error under low SNR 
conditions. The methods are meant primarily for synchronization in a FDMA satellite system where 
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matched filtering and demodulation is performed by a SAW Founer transform device followed by an 
envelope detector. 

Two implementations, denoted the two-tone and early-late filter methods, arc analyzed. For each 
implementation, the time error estimate is derived using either the first and second order moments of the 
received envelope, or the second and fourth order moments. A unique method using three second order 
moments for the eariy-late filter implementation is included. Results are given to show the effect of SNR 
on the mean and standard deviation of the estimates for a range of actual time error 
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SESSION MA2 


DETECTION THEORY I 


Asymptotic Efficiencies in Multiple-Access Channels 

S. Y. Miller and S. C Schwartz Department of Electrical Engineering, Princeton University, Princeton, 
NJ 08540 

The evaluation of the performance of a central multiuser receiver is an important issue in multiuser 
communication. In some cases, where explicit analytical results are unavailable, a fruitful approach is to 
resort to the (asymptotic) vanishing noise assumption which yields closed form results. In all cases, it is 
interesting to compare the resulting performance for any of the users to the performance of a single user 
receiver in a related single user channel. The possible multipie access degradation is then expressed in 
terms of the efficiency of the multiuser receiver. In this paper asymptotic efficiencies of multiuser 
sequence detectors arc evaluated employing an approach which relics on a large deviation result of H. 
Chemoff. A strong relationship between the results of this approach, applied to the Gaussian multiple 
access channel, and existing results for optimum and sub-optimal muluuscr detectors is indicated The 
examples given illustrate the possible general applicability of the Chemoff result to the calculauon of 
multiuser asymptotic efficiencies. 


Importance Sampling: A Robust Statistics Approach 

Geoffrey Orsak and Bchnaam Aazhang Computer and Information Technology Institute, Department of 
Electrical and Computer Engineering, Rice University, Houston, TX 77251-1892 

The problem of estimating expectations of functions of random vectors via simulation is 
investigated. Monte Carlo simulations, also known as simple averaging, have been employed as a direct 
means of estimation. A technique known as Importance Sampling can be used to modify the simulation 
via weighted averaging in the hope that the estimate converges more rapidly to the expected value than 
standard Monte Carlo simulations. The fundamental problem in Importance Sampling is to determine the 
appropriate density function for the underlying random vanabie in ihe simulation Since the 
unconstrained optimal solution to this problem is degenerate, suboptimal solutions have been of the form 
of a scaled, linearly shifted or exponentially tilted version of the original density In this paper, we denvc 
a constrained optimal solution to the problem of minimizing the variance of the Importance Sampling 
estimator. This is done by finding the distribution which is "closest" to the unconstrained optimal solution 
in the Ali-Sdvey sense. The solution from the constraint class is shown to be the least favorable density 
function in terms of Bayes risk against the optimal density funcuon. Examples of constraint classes, 
which include e -mixture, will show that the constrained optimal solution can be made arbitrarily close to 
the optimal solulioa Applications to estimating probability of error in communication systems will be 
given 


Performance of Optimal Non-Gaussian Detectors 

Dor. H. Johnson and Geoffrey Orsak Computer and Information Technology Institute, Department of 
Electrical and Computer Engineering, Rice University, Houston, TX 77251-1892 

The optimal procedure for detecting the presence of discrete-time signals in additive noise can be 
derived from the likelihood ratio test When the noise has statistically independent, identically distributed 
components, the dependence of the detector’s performance on signal charactcnstics can be related to the 
Kullback-Leibler (KL) distance between the distributions governing the hypotheses. Performance 
predictions based on the central limit theorem are shown to be poor approximations to the true 
performance. Performance of the optimal detector has long been known to decrease exponentially with 
increasing Kullback-Leibler distance, symmetric noise amplitude distributions yield a symmetric 
dependence on the difference between the signals' amplitudes at each time index. Small-signal (locally 
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optimal) detection performance is shown to depend on signal energy in proportion to the Fisher 
information for location. When a distance measure can be defined, performance depends on a different 
measure than used in the detector with one exception (the Gaussian). Large-signal performance depends 
on the signal waveform with the most dramatic contrast between signal sets compnsed of constant- 
difference signals and of signals differing in only one value. 


Computing Distributions from Moments Using Padd Approximants 

James A. Ritcey and Hamid Amindavar Department of Electrical Engineering, University of 
Washington, FT-10 Seattle, WA 98195 

The evaluanon of signal detecuon schemes is often hampered by the inability to accurately compute 
tail probabilities. In fact, often only the moments of the test statistic can be computed and we seek to 
invert this information to obtain cumulative distribution functions One approach is to determine the 
moment generating function, through a truncated power series expansion. In this paper we examine the 
performance of Padd approximants to approximate the m g.f., and we carry out the inversion using the 
method of residues. The diagonal Padd approximants are compared to a two-point Padd method which 
matches the c.d f around zero and infinity. The advantage of this technique is that only low order 
moments are required. This is most important when the moments are estimated, and higher order 
moments are significantly m error. The method is applied to some common problems in signal detection 
theory. 


A Mcmoryless Grouped-Data Nonparamelrie Sequential Detection Procedure 

M. M Al-Ibrahim and P K. Varshncy Department of Electrical and Computer Engineering, III Link 

Hall, Syracuse University, Syracuse, NY 13244-1240 

A nonparametnc sequential testing procedure for the detection of constant signal in additive 
symmetric noise is proposed and analyzed The nonparametnc test is obtained by first quantizing all the 
observations into their signs, and then applying the Lee-Thomas modified sequential procedure to the 
quantized observations, or their sufficient statistic. The procedure is also extended to a dislnbuted system 
consisting of two local detectors and a global decision maker A simple truncation scheme is considered 
and is shown to maintain the nonparamctric property while efficiently limiting the random test duration. 
Numerical results arc presented to illustrated the performance gam and to demonstrate the simplicity of 
the test structure 


A Simple Approach to the Design of Decentralized Bayesian Detection Systems 

W. Hashlamoun and P K Varshncy Dept of Electrical & Computer Engineering, Ill Link Hall, 

Syracuse University, Syracuse, NY 13244-1240 

This paper considers the design of optimal decentralized Bayesian detection systems A simple 
approach based on a version of Kolmogorov vanational distance is presented The design complexity of 
our approach is much less than the previous methods. A two sensor example is given for illustration 


Decision Agreement and Link Usage in Distributed Detection Systems with Feedback 

Sam. Alhakecm, R. Snmvasan, and P. K Varshncy Electrical and Computer Engineering Department, 

Syracuse University, 111 Link Hall, Syracuse, NY 13244-1240 

We consider a decentralized detection system with feedback The issues of agreement probability 
and link usage in this feedback structure are investigated. Using some properties of this structure, we 
derive asymptotic results for the agreement probability The second issue addresses the use of 
communication links between detector and the fusion center. We propose and analyze two protocols to 
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reduce the link usage. We derive equations for the average number of links needed under both hypothesis 
Hi and H 0 , and study the asymptotic performance. 


A Converse Theorem for a Class of Multiterminal Detection Problems 

Hossam M. H Shalaby and Adnan Papamarcou Electrical Engineering Department and Systems 
Research Center, University of Maryland, College Park, MD 20742 

_ _We discuss the problem of testing a bivanate hypothesis H. Pxy against a simple alternative 
H : Pxy on the basis of i.i.d. pairs of discrete-valued observations (X,Y) We assume that the Y data are 
directly accessible to the decision maker or detector, while the X data are compressed at asymptotically 
zero rate. For Pxy > 0, we show that the error exponent of the optimal test of level e (i e., the test that 
minimizes the type 11 error probability subject to the type I error probability not exceeding e) is 
independent of £ for any e e (0,1). In doing so we generalize a previous result by Han that demonstrated 
’he constancy of the error exponent for e ranging in a subinterval of (0,1) Our result readily extends to 
hypothesis tesnng involving multivanate distributions on higher dimensional product spaces. 
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SESSION MA3 


NEURAL NETWORKS I 


Nested Neural Networks and Their Codes 

Yoram Baram Dept, of Elec. Engrg., Technion, Israel Institute of Technology, Haifa, 32000, Israel (40 
min.) 

Neural networks can be viewed as encoders that employ sets of stored neural patterns as codes. 
Fully connected binary networks have been shown to provide low storage capacity and error correction 
capability. Networks composed of nested layers of small subnetworks are defined and shown to retrieve 
those permutations of the subcodes stored in the subnetworks that agree in their common bits. The 
resulting code sizes are considerably greater than those allowed by fully connected networks and the 
number of intemeural connections is considerably smaller. Nested codes defined on subcodes consisting 
of two subwords are shown to be stable states of the network and to provide a relative error correction 
capability near 0.5 per subnetwork. Orthogonality of the subwords is shown to guarantee that their 
permutations are stable states of the nested network and that errors of relative size r per subnetwork will 
be corrected if the subcode stored in each subnetwork is of size smaller than 1 1(2r). For randomly stored 
subcodes, the nesting property is shown to increase the probability of nondivergence and of error 
correction. In nondiscriminatory storage of such subcodes, stability and error correction capability can be 
guaranteed if and only if the subcodes stored in the subnetworks arc of size 2 at most. Nested codes and 
their sizes are characterized for general subcodes and for specific orthogonal ones. 


On the Number of Spurious Memories in the Hopfieid Model 

Jehoshua Bruck and Vwani P. Roychowdhury IBM Almaden Research Center, 650 Harry Road, San 
Jose, CA 95120-6099 and Information Systems Laboratory, Stanford, CA 94305 

We show that the outer-product method for programming the Hopfieid model can result in many 
spurious stable states-exponential in the number of vectors that we want to store-even in the case when 
the vectors are orthogonal. 


Some Statistical Convergence Properties of Artificial Neural Networks 

Andrew R. Barron Departments of Statistics and Electrical & Computer Eng., University of Illinois, 725 
S. Wright Street, Champaign, IL 61820 

Convergence properties of empirically estimated neural networks are examined. In this theory, an 
appropriate size feedforward network is automatically determined from the data. The networks we study 
include two and three layer networks with an increasing number of simple sigmoidal nodes, multiple 
layer polynomial networks, and networks with certain fixed structures but an increasing complexity in 
each unit Each of these classes of networks is dense in the space of continuous functions on compact 
subsets of d-dimensional Euclidean space, with respect to the topology of uniform convergence. In this 
talk we show how, with the use of an appropriate complexity regularization criterion, the statistical risk of 
network estimators converges to zero as the sample size increases. Bounds on the rate of convergence are 
given in terms of an index of the approximation capability of the class of networks. 

It’s OK to be a Bit Neuron 

Santosh S. Venkatesh Moore School of Electrical Engineering, University of Pennsylvania, 
Philadelphia, PA 19104 

We investigate computational and learning attributes in formal neurons when the neural weights are 
constrained to be binary (bit neurons). We demonstrate formally that there is little computational loss in 
eschewing real interconnections in favor of binary links: with binary weights the achievable 
computational capacity is a much as one half that with real interconnections. Analogous results hold for 
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learning weights from instances of a problem. Learning real weights for a McCulloch-Pitts neuron is 
equivalent to linear programming and can hence be done in polynomial time Efficient local learning 
algorithms such as the Perceptron Training rule further guarantee convergence in finite tune. The problem 
becomes considerably harder, however, when it is sought to leam binary weights; this is equivalent to 
integer programming which is known to be NP-complete. In the second part of this paper we present a 
new family of probabilistic binary learning algorithms. These algonthms have low computational 
demands and are essentially local in character. Rapid mean convergence times are demonstrated. 


On Reliability and Capacity in Neural Computation 

Santosh S. Venkatesh and Demetri Psaltis Moore School of Electrical Engineering, University of 
Pennsylvania, Philadelphia, PA 19104, and Department of Electrical Engineering, California Institute of 
Technology, Pasadena, CA 91125 

We investigate the computing capabilities of formal McCulloch-Pitts neurons when errors are 
permitted in decisions. Specifically, we investigate the following. 

Epsilon Capacity: In a fully interconnected recurrent network of formal neurons what is the 
maximum number of memones that can be stored given a presenbed level of recall error tolerance? 

We show the following rigorous results given 05c < 1 12, a fractional error tolerance 

• Consider a random m-set of points in n-space and a set of associated binary decisions (or 
classifications) to be made on these points by a single formal neuron, the sequence 2n/(l-2e) is a 
threshold function for the property that there exists a choice of synaptic weights for the neuron such 
that no more than (essentially) em random decision errors is made; further, there is a function 
1 S/r £ <505 such that if m exceeds 2 ic c /i/ 1(1-2£) then Uiere is asymptotically no choice of 
synaptic weights for which a neuron makes fewer than cm decision errors on the m-set of inputs 

* Consider a random m-set of memones to be stored in a fully interconnected recurrent network of n 
neurons the sequence 2n/(l -2c) is a threshold function for the property that there is a choice of 
neural interconnccuons such that there are no more than (esscnually) en random bit errors in recall 
of any memory, if m is chosen larger than 2tc e nl(\-2e) then for no choice of interconnections is 
any memory confined within a Hamming ball of radius en 

For small e the function K e is close to 1 so that, informally, we can specify ni-scts of points (memones) 
as large as 2nl(\-2e) (but not larger) and obtain e-reliablc decisions (c-error tolerance) for some suitable 
choice of synaptic weights. 


Complexity of a Finite Precision Neural Network Classifier 

K Siu, A. Dcmbo, and T Kailath Information Systems Laboratory, Stanford University. Stanford, CA 
94305 

A ngorous analysis of the finite precision computational aspects of a neural nctwoik as a pattern 
classifier via a probabilisuc approach is presented. Even though there exist negative results on the 
capability of the perceptron, we show the following positive results, given n pattern vectors, each 
represented by cn bits where c > 1, that arc uniformly distributed, with high probability the perceptron 
can perform all possible binary classificauons of the patterns Moreover, the resulting neural network 
requires a vanishingly small proportion OQognln) of the memory that would be required for complete 
storage of the patterns Further, the perceptron algomhm lakes 0(n 2 ) arithmetic operations with high 
probability, whereas oilier methods such as linear programming take 0(n 35 ) in the worst case We also 
indicate some mathematical connections with VLSI circuit testing and the theory of random matnccs 
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The Information Provided by a Linear Threshold Function with Binary Weights 
Rodney M. Goodman, John W. Miller, and Padhraic Smyth Dept, of Electrical Engineering (116-81), 
California Institute of Technology, Pasadena, CA 91125, and Communication Systems Research, Jet 
Propulsion Laboratories 238-420,4800 Oak Grove Drive, Pasadena, CA 91109 

The J measure, originally used to measure the expected change in information provided by a 
production nde, is applied to measure the information provided by a linear threshold function (LTF) with 
binary (0,1) weights. This LTF with binary weights implements the logical function meaning "X of these 
N inputs are ON", called an (X ofN) rule. For example 3 (1 of AO nde is an OR rule, and an (At of AO rule 
is an AND rule. In terms of the memory required to describe a basic logical unit, the (X of N) nde lies 
between the general LTF and the logical AND and OR functions. Logical AND and OR functions are 
used in production rule systems as a compact representation of domain knowledge which can be 
understood easily by humans Efficient algonthms exist for searching through a database for informative 
conjunctive production rules. This paper extends this work to show how sets of informative (X of N ) rules 
can be found, and shows why the (X of A 1 ) rules can be more powerful than conjunctive and disjunctive 
roles as unit of knowledge. 


II 


SESSION MA4 


DATA COMPRESSION 


Almost Sure Data Compressioii for Processes 

Paul C. Shields Dept of Math., U of Toledo, Toledo , OH 43606 

Universal coding for processes was first discussed in the landmark papers of Davisson, Lynch, and 
Fittmghof, and generalized by many subsequent authors to obtain asymptotic optimality in terms of 
expected value criteria, convergence in probability, or L‘ convergence. By extending to the rate- 
distoition setting a new entropy estimation technique of Omstein and Weiss, ("How sampling reveals a 
process,” Annals of Prob. (to appear)), ve show how to construct a universal sequence of codes that is 
asymptotically optimal in the almost sure sense. The codes use the empmeal distnbution of 
nonovcrlapping /t-blocks in a sequence of length re, all that is is required is that n>4 a , where A is the 
alphabet size Some methods for computer implementation of these ideas will also be discussed (This 
talk is based on joint work with D. S Omstein that will appear in the Armais of Probability) 


Finite Memory Modeling of Individual Sequences with Applications to State Estimation and 
Universal Data Compression 

Marcelo 1 Weinberger, Abraham Lcmpel, and Jacob Ziv Techmon - Israel Institute of Technology, Haifa 
32000, Israel 

This paper deals with the estimation and the universal compression of discrete sources The notion 
of stability of a word relative to an individual sequence over a finite alphabet is defined. In case of 
sequences emitted from a probabilistic finite memory source, the concept of stability is shown to be 
closely related to the estimation of the set of states of the source 


A Fast Construction Algorithm of Coding Tree for Variable-Length Data-Compression Coding 
with Fidelity Criterion 

Hisashi Suzuki and Suguru Anmoto Department of Mathematical Engineering and Information Physics, 
Faculty of Engineering, University of Tokyo, Bunkyo-ku, Tokyo 113, Japan 

The problem of vanaote-length data-compressing coding with A-distortion aims to realize some 
code such that: as the message length approaches infinity, the expectation of the codeword length divided 
by message length approaches the A-distortion rate, and, the probability that the distortion between 
messages and their reproductions may not exceed A approaches I This paper proposes a top-down 
construction algonthm of balanced tree with a pointer indexed by an arbitrary-given distortion measure, 
and applies it to a method of variable-length data-compression coding. The mam specifications of the 
obtained coding method are the following: the computation time for constructing a coding tree per 
codeword is 0(k), and that for each of encoding and decoding per codeword is also 0(k), where k 
denotes the message length; the expectation of the codeword length divided by message length 
approaches the A-distortion rate; the probability that the distortion between messages and their 
reproductions may not exceed A-distortion rate; the probability that the distortion between messages and 
their reproductions may not exceed A approaches 1 


Optimum Bit Allocation via the Generalized Breiman, Friedman, Oishen, and Slone Algorithm 
Eve A. Riskin and Robert M Gray Information Systems Laboratory, Stanford University, Stanford, CA 
94305 

An extension of Breiman, Friedman, Oishen, and Stone’s algorithm for optimal pruning in tree- 
structured classificauon and regression is appbed to bit allocation It uses a simple tree model to 
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deallocate bits from classes of a coder, resulting in a set of vanable rate codes of different average rates 
When applied to classified vector quantization, it leads to a 3.4 dB gain in the signal-to-noise raao of a 
magneuc resonance image coded at 1 bit per pixel over fixed rate classified vector quantization at the 
same rate. 


Asymptotic Optimality of a Universal Variable-to-Fixed Length Binary Source Coding Algorithm 
Tjalling J. Tjalkens and Frans M. J Willems Eindhoven University of Technology, P 0 Box 513, 5600 
MB Eindhoven, The Netherlands 

The modified Lawrence algonthm was introduced by the authors in 1988 as a universal binary 
variable-to-fixed length source coding algonthm. Here we investigate its asymptotic performance. For M 
(the segment set cardinality) large enough, wc show that the rate R (p) as funcuon of the source parameter 
p satisfies 

forO <p < 1, where A(-) is the binary entropy function 

In addition to this, wc prove that no codes exist that have a better asymptotic performance, thereby 
establishing the asymptotic optimality of our modified Lawrence code 

The asympiouc bounds show that universal vanable-to-fixed length codes can have a significantly 
lower redundancy lhan universal fixed-to-variable length codes with the same number of code words 
An Entropy Constrained Quantization Approach for a Source Characterized by a Random 
Parameter 

Chein-I Chang and Lee D. Davisson Department of Electrical Engineering, University of Maryland, 
Baltimore County Campus, Baltimore, MD 21228, and Electrical Engineering Department, University of 
Maryland, College Park, MD 20742 

The problem of quanuzing a source charactcnzed by a random parameter is considered. The method 
proposed in this paper is an cntropy-constramed approach. The idea is that for a given source output 
partition we use a source matching algonthm to find the minimax entropy for the class of sources 
generated by the random parameter, then minimize the mean squared quantization error subject to the 
entropy constraint which is obtained by maximizing the minimax entropy over all possible partitions of 
me source output space. Since it is known mat the uniform quantizer achieves me optimum performance 
at high-bit rates, asymptotic results for me proposed approach can be iurthcr obtained by replacing me 
optimum quantizer with me uniform quantizer. 


The Redundancy Theorem and New Bounds on the Expected Length of the Huffman Code 
Raymond W Yeung AT&T Bell Laboratories, Crawfords Corner Road, Holmdel, NJ 07733-1988 

We introduce me Redundancy Theorem as a tool to lower bound me expected length of prefix 
codes. It is shown mat virtually all me previously known lower bounds of me expected length of me 
Huffman code can be obtained via applications of me Redundancy Theorem, and we demonstrate an 
application of me theorem which yields new lower bounds. We also obtain a new upper bound of the 
expected length of me Huffman code which depends on me entropy of me source and me two smallest 
probabilities of me distribuuon. 


Alphabetic Codes Revisited 

Raymond W. Yeung AT&T Bell Laboratories, Crawfords Corner, Holmdel, NJ 07733-19S8 

An alphabetic code for an ordered probability distribution <p t > is a prefix code in which pt is 
assigned to the km codeword of me coding tree in the left-to-nght order. This class of codes is applied to 
binary test problems. We denve me characteristic inequality for alphabetic codes which is analogous to 


A- 


13 




the Kraft Inequality for prefix codes. With this inequality, we are able to unify and enhance many 
previous results on alphabetic codes. We discover that when <p k > is in ascending or descending order, 
the expected length of an opnmal alphabetic code for <p k > is the same as that of a Huffman code for the 
unotdered distribution { p k ). We also prove two new lower bounds of the expected length of an opnmal 
alphabetic code, and propose a simple method for constructing good alphabetic codes when optimality is 
not cnncal. 


SESSION MAS 


CHANNEL CAPACITY 


On the Capacity of a Spectrally Constrained Poisson-Type Channel 

Amos Lapidoth and S. Shamai (Slutz) Department of Electrical Engineering, Techmon - Israel Institute 
ofTechnology, Haifa 32000, Israel 

The classical direct detection optical channel is modeled by an observed Poisson process of 
intensity A, + A„ wher A, is the information carrying waveform and A„ stands for die dark current 
intensity. An upper bound on the capacity of this channel subjected to peak power 0< A, <A, average 
power E(X,) = a and covariance (spectral) constraints V'(r) = £'(A,A, +I )-£' J (A,) is denved using the 
Kabanov-Davis approach that inte. relates mutual information with nonlineai filtenng in the Poisson 
regime. The resultant bound incorporates known results of optimal linear filtenng in the Poisson regime 
and coincides with the known (peak and power constrained only) capacity as the spectral constraints arc 
relaxed. 

This new bound can be interpreted as the Poisson channel equivalent of Shannon's information 
integral 'AjZ, log [ I -t- ZS’ I O r )/A r 0 ]£$ r which upper bounds the capacity of the additive white Gaussian 
channel, where S x (f) and N„l2 arc respectively the input signal and noise power spectral density 
functions. 

Lower bounds on the capacity under second moment and stnet bandwidth constraints arc found 
using for the modulating waveform A,, a special class of pulse amplitude modulated signals satisfying the 
required constraints 


Some Results on Zero-Error Capacity Under List Decoding 

Erdal Ankan Department of Electrical Engineering, Bilkent University, PO Box 8 , Maltepe, Ankara 
06572, Turkey 

We address a number of questions recently raised by Elias. Let M N (L) be the size of a maximal 
zero-error list-of-A code with blocklcngth N Let Cod-) be the zero-error list-of-A capacity defined by 

C 0 (L) = !im inf logAfv(A) 

N-tm N 

Elias shows that 


hm Cq{L) = Cof 

where Cor is the zero-error capacity of the same channel with feedback We prove the following results. 

1. Computing Cor )S an NP-complctc problem 

2. For all L a 1 and JV 21,, 

iVWf-)Si.2 C 'Af*(L)J 

>From this we obtain, e.g., that, for die 3/2-channcl discussed by Elias, M 4 (2) <9, answering one of 
his questions. 

3. For all A 21, A21, and ISASA, 

M N (L)>M N+n {K) 

where n is the smallest integer such that Af„(/f)2 L. 

The question of how ught these bounds are is also discussed 
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The Capacity-Cost Function of a Noiseless Channel with Several Cost Constraints 

Robert J McEliece and Lada Popovid Department of Electrical Engineering, California Institute of 

Technology, MC 116-81, Pasadena, CA 91125 

A noiseless channel is usually represented by a directed graph whose vertices are the channel 
symbols, the allowable input sequences being the walks in the graph. Some properties of the channel can 
be modeled by assigning to each edge a vector of c e R" and requiring that only those walks are allowed 
whose cost-per-edge does not exceed, componentwise, some given vector w. 

We define the Capacity-cost function 

C(w) = lim sup 

n —n 

(where M(n,vt) is the number of walks of length n and per-edge cost <w) and show that it is equal to the 
maximal entropy of a Markov chain which is supported by the graph and whose expected cost per 
transition does not exceed w. We also derive an expression for C (w), thus extending the result of Justcsen 
and Hoholdt to muludimensional costs. 

Using the above results, we obtain a formula for the cxponenual decay rate 

«-*- n 

of the probability that a sequence has less than a specified per-transition cost on a given Markov chain 
with real costs assigned to the transitions 


A New Upper Bound on e -Capacity 

Michael L Horug and Prakash Narayan Bellcore, 445 South Street, Morristown, NJ 07960 and 
Department of Electrical Engineering and Systems Research Center, University of Maryland, College 
Park, MD 20742 

Suppose that two outputs of a linear, time-invariant channel are disunguishable at the receiver if and 
only if they arc separated in Li norm by e The inputs to the channel arc assume to be power limited, and 
are nonzero only on the finite time interval (0, T1 Let N ma (T, s) be the maximum number of 
distinguishable outputs for given T, e >0 The e-capacity of the channel is defined as 
C 6 = lim Iog 2 [A r rt>lx (7', e )}/T bits/sccond. It has been shown by Forys and Varaiya that C 6 is upper 

T 

bounded by the Shannon capacity, Cy, of a channel consisting of the original channel followed by 
additive Gaussian noise with variance e J /4, but otherwise having arbitrary spectral density We show 
that the noise spectral density that minimizes C s is proportional to the power spectral dcasity of the input. 
It is also shown that the resulting upper bound on C t is less than or equal to the Shannon capacity of a 
channel consisting of the original channel plus an additive noise source (not necessarily Gaussian) with 
variance e 2 /4, but otherwise having arbitrary statistics Numerical results are presented for models of 
subscriber loop channels that show the new upper bound is significantly better than the upper bound 
previously derived by Root. 


A Lower Bound on the Capacity of Primitive Binary BCH Codes Used in Gaussian Channel with 
Discrete Time 

Dejan E. Lazid and Vojin Scnk Universitat Karlsruhe, Insntut fur Algorithmen und Kognitive Systeme, 
D-7500, Karlsruhe 1. Fed. Rep of Germany, and University of Novi Sad, Computer Science, Control and 
Measurements Institute, Novi Sad, Yugoslavia 

Since the BCH codes are not asymptotically good, it was long believed that long BCH codes do not 
behave well in a communication channel This conjecture was based on the assumption that the ratio of 
the minimum Hamming distance of the code to the code length exhibits the dominant influence on the 
error probability for any signal-to-noise rauo and any dimension (code length). This conjecture is 
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disproved in this paper, showing that in the additive white Gaussian noise channel, for large enough 
dimension and low signal-to-noise ratio SNR the Hamming distance and the Euclidean distance that is 
monotomcally related to it are of no influence on the error probability, provided that the multiplicity of 
code words on the distance is below a certain bound. The capacity of long pnmiuve binary BCH codes 
used in Gaussian channel with discrete time is, using this result, lower-bounded by 

R =l-ld(l+2- SNKl2h2 ) 

~BCH 

i.e., for a given SNR very long primitive binary BCH codes of rate R can be safely used in the 

"BCH 

Gaussian channel, provided that true minimum-distance decoding is used instead of the usual hard- 
decision bounded-distance decoding method. 


Zero-Error Capacities and Very Different Sequences 

G. Cohen, J. Komer.and G. Simonyi ENST, 46, rue Barrault 75634 Pans Cedex 13, France, and 
Mathematics Institute of HAS, 1364 Budapest, FOB 127, Hungary 

Perfect hash functions, superimposed codes as well as some other fashionable questions in computer 
science and random-access communication are special cases of early-day information theoretic models in 
the zero-error case 

A new class of problems m asymptotic combinatoncs can be formulated as the determination of the 
zero-eTor capacity of a class of discrete memoryless channels. (This model is also known as the 
compound channel). We solve an interesting class of these problems using our recent results in polyhedral 
combinatoncs. 


Capacity of the Gaussian Arbitrarily Varying Channel 

Imre Csiszdr and Prakash Narayan Mathematical Institute of the Hungarian Academy of Sciences, H- 
1364 Budapest, POB 127, Hungary, and Electrical Engineering Deparment and the Systems Research 
Center, University of Maryland, College Park, MD 20742 

The Gaussian arbilranly varying channel with input constraint T and state constraint A admits input 
sequences x = (i i,. . ,r„) of real numbers with l/n jr? < T and state sequences s = (r | ,. ,r„) of real 

numbers with 1/n Js?SA, the output sequence being x+s+V where V = (Vj,. ., V,) is a sequence of 
independent and identically distnbuted Gaussian random variables with mean 0 and vanancc <7 . We 
prove that the capacity of this arbitranly varying channel for deterministic codes and the average 

probability of error entenon equals 1 h log (1 + — —r) if A < Tand is 0 otherwise. 

A + cT 
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SESSION MA6 


CODING THEORY I 


Exponential Error Bounds for Randomly Modulated Codes on Gaussian Channels with Arbitrarily 
Varying Interference 

Brian Hughes and Tony G. Thomas Department of Electrical and Computer Engineering, The Johns 
Hopkins University, Baltimore, MD 21218 (40 min.) 

The Gaussian atbitranly varying channel (GAVC) models a channel corrupted by thermal noise and 
by an unknown interfering signal of bounded power In this paper, we present upper and lower bounds to 
the best error probability achievable on this channel with random coding. The asymptotic exponents of 
these bounds agree in a range of rates near capacity and at rate zero. The exponents are universally larger 
than the corresponding exponents for the discrete-time Gaussian channel with the same capacity. 

We further show that the opumal exponents can be achieved by a restricted kind of random code 
comprised of a deteiminisuc encoder/decoder in cascade with an independent linear random 
modulator/demodulator Moreover, the decoder can be taken to be the minimum Euclidean distance rale 
at all rates less than capacity. Connections to spread-spectrum modulation are discussed 

Spectral Lines of Codes Given As Functions of Finite Markov Chains 
Hiroshi Kamabe Department of Electronics, Mie University, Tsu 514, Japan 

We analyze the spectral line of a memoryless function of a finite Maikov chain. We expect that the 
results of this paper will have applications in fiber optics A mcmorylcss funcuon of a finite Markov chain 
is given by a tnple (G.P.y), where G is a directed graph, P is a transition probability matrix compatible 
with G and y is a complex valued function of the vertex-set of G. We prove that for Gy and a positive 
real number c the following two conditions are equivalent: (1) for any P the spectral line of (G.P.y) at a 
rauonal submuitiplc / of the symbol frequency is equal to (not less than, respectively) c; (2) for any loop 
of G, if its length is L, then the absolute value of the running digital sum of the sequence of values of y of 
the vertices along the loop at /, is equal to (not less than, respectively t'ic Wc also characterize these 
conditions in terms of coboundary functions introduced by B Marcus and P. Siegel 


On the Construction of Statistically Synchronizable Codes 

R. M Capocelli, A. De Santis, L. Gargano, and U. Vaccaro Dipariimento di Matematica, Umversita'di 
Roma, 00185 Roma, Italy, IBM T J Watson Research Center. P 0 Box 218, Yorktown Heights, New 
York, 10598, and Dipariimento di Informatica ed Applicationi, Umversita' di Salerno, 84100 Salerno, 
Italy 

We consider the constracuon of almost-opumal statistically synchronizable codes for arbitrary 
alphabets and finite sources. We show that it is always possible to obtain a code possessing a 
synchronizing sequence and presenting an increase on the optimal average code word length not greater 
than the lowest probability associated to a codeword. Moreover, we give an efficient method to construct 
codes having a synchronizing word. These codes are almost-optimal, in the sense of a small average 
length, and present hign synchronization capability. All previous proposed solutions were rcstneted to 
particular sources and considered only a binary alphabet. 
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‘1’-Ended Binary Prefix Codes 

Renato M. Capocell! and Alfredo De Santis Dipartimento di Matematica, University di Roma, 00185 
Roma, Italy and IBM T. J Watson Research Center, P O. Box 218, Yorktown Heights, New York, 10598 
Binary prefix codes with the constraint that each codeword must end with a ‘ 1* have been recently 
introduced by Berger and Yeung. We analyze the perfomiances of such codes by investigating their 
average codeword length. In particular, we show that a very simple strategy permits the construction of a 
•‘l'-cnded binary prefix code whose average codeword length is within H+ 1 for any discrete source with 
entropy H. We also prove a tight lower bound on the optimal average codeword length in terms of H and 
the minimum probability of the source. Finally, we discuss the problem of finding an optimum code. 


Mathematical Models for Block Code Error Control Systems on Renewal Inner Channels 
D. R. Oosthuizen, H. C. Ferreira, and F. Swans SA Transport Services, Telecommunication and 
Laboratory for Cybernetics, Rand Afrikaans University, PO Box 524, Johannesburg 2000, South Africa 
We investigate the average event rate and statistical distnbution of the block error detection, 
correction and misdetecuon events of block codes on discrete renewal inner channels, when the latter arc 
represented with partitioned Maikov chains as proposed by Fritchman. We show that the underlying 
statistical structure of the cvems under consideration can be represented with outer channel models, 
which are also partitioned Markov chains, similar to the inner channel models Analyucal procedures to 
determine the parameters of such an outer channel model, as a function of the inner channel model 
parameters, arc presented. The results of extensive computer simulations to venfy the analytical 
procedures, are also presented The application of the outer channel models in the design of error control 
systems, such as systems with repetition of codewords and rcjecuon of codewords with detected errors, as 
well as ARQ systems, is investigated. 


Run Length Coding with Spectral Lines 

Kenneth J Kcrpez Bell Communications Research, Morristorn. NJ 07962-1901 

Codes arc considered that map information into a binary ran length limited sequence with spectral 
lines. The codes provide pilot tones for tracking m magnetic or optical recording. The coded sequences 
are generated by variable length state transition diagrams that are periodic with period p A sufficient 
condition that the code has spectral lines is that the square of the state transiuon diagram has period 2 p 
The poles in the spectrum or. the unit circle occur at the p' k roots of unity; the lines occur at the 
corresponding frequencies. A deterministic component in the coded sequence has a periodic 
autocorrelation, which transforms to a line spectrum A method of computing the power in the line 
spectrum from stationary probabilities of the state diagram is shown Relations arc given that prove that 
as the power in the line spectrum or the number of lines increases, the information capacity of the code 
decreases Two families of graphs are shown that produce codes with spectral lines. 


Error Fret Coding on Chinese Characters 

Rong-Hauh Ju, I-Chnng Jou, Mo-King Tsay, and Kuang-Yao Chang Teteiommunaation Laboratories, 
Ministry of Communications, P.0 71, Chung-U, Taiwan, R.O C ., and Institute of Information & 
Electronics, National Central University Chung-Li, Taiwan, ROC 

Dot pattern representation is the most popular method to represent Chinese characters. However, the 
storaging memory of these Chinese character patterns is too large to be processed well. In this paper, the 
characteristics of Chinese patterns are introduced and analyzed first, then some data-compression 
methods have been tried, aiming at the smallest amount of data storage The compression algorithms 
(preprocessing, statistics of source symbols, encoding), which are described by three passes, will be 
discussed by their compression ratio, entropy, and complexity. We find out that the compression method 
(prediction, subblock, anlhmetric coding) will achieve a best compression ratio of about 25% and save 
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understand a possible bound on compression on Chinese characters with vanous resolutions. 


Random Modulation/Coding Problems for a General Channel 

Shi vi Shcn Department of Mathematics, Nankin University, Tianjin, PR China 
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SESSION MA7 


TRELLIS CODING I 


Coherent and Differentially Coherent Trellis Coded Modulation on Channels with Correlated 
Time-Selective Fading 

Christian Schlegel Communications Group, ASEA BROWN BOVERI Corporate Research, 5405 Baden, 
Switzerland 

Trellis-coded modulauon (TCM) on correlated fading channels, descnbed by the wide sense 
stanonary uncorrclatcd scatterer (WSSUS) model, is studied For this channel model the two code error 
probability can be expressed as the probability that attain Gaussian quadrauc forms exceed zero. It is 
shown how the event error probability P e , upper bounded by the averaged union bound, can be calculated 
with an efficient algonthm using quasi-regulanty properties of the codes used. It is further shown that in 
the limiting case of no interleaving, the Euclidean distance spectrum can be used to evaluate P e , while for 
full interleaving, the effective length/minimum product spectrum dclemunes perfomnance. 

It is further shown how differential TCM on correlated fading channels, in conjunction with the 
suboptimum Euclidean distana metric, can be heated analytically Results show that differential TCM 
suffers a degradation between 2.5 dB and 5 dB with respect to coherent TCM 

In a refinement of the analysis, non-ideal estimation of the channel impulse response in the coherent 
case and co-channel as well as adjaant channel interference are included 


Analysis of the Error Performance of Trellis-Coded Modulations in Rayleigh Fading Channels 
Jim Cavers and Paul Ho School of Engineering Science, Simon Fraser University, Burnaby, B C, 
Canada V5A 1S6 

Trcllis-C',dcd Modulauon (TCM), when combined with interleaving, is known to give good error 
performance in fading channels Previously, though, the only analyucal guide has been an upper bound on 
the pairwise error event probability, which could be very loose over the range of signal to noise rauo of 
interest In contrast, this paper presents an exact expression for the pairwise error event probability of 
TCM transmitted over Rayleigh fading channels. It includes PSK and multilevel QAM codes, as well as 
coherent and partially coherent (e g., differential, pilot tone, etc.) detection We have found that a good 
esumatc of the bit error probability can be obtained by considering only a small number of short error 
events This implies digital computer simulauon can be avoided during the link design proass. We study 
several coded modulation schemes this way. Among the results are the fact that compared to uncoded 
modulation, TCM provides a significant improvement in the error floor when detected differentially, and 
an asymmetry in the pairwise error event probability for 16QAM 


Trellis Coded MPSK Modulation with an Unexpanded Signal Set 

Shalim S. Penyalwar and S. Fleishcr Department of Electrical Engineering, Technical University of 
Nova Scotia, Halifax, NS B3J 2X4 

In this paper, a scheme for transmitting two channel symbols per trellis branch is proposed, wherein 
the output bits from a rate mlm +1 encoder are used to select pairs of symbols from a 2" , -ary 
(unexpanded) MPSK symbol set. The proposed set partitioning technique simultaneously optimizes the 
symbol assignments to the trellis branches for performance on the AWGN channel (maximize d 2 (free)) 
and the fading channel (maximize diversity and product of branch distances along error event path of 
shortest length). The throughput rate is m/2 bps/Hz. For a given throughput rate, the complexity of the 
scheme (measured by symbol multiplicity, number of parallel paths, and number of states) lies between 
that of the TCM and MTCM schemes, and the performana gams achieved on the AWGN and fading 
channels are equal to or higher than those of MTCM schemes We demonstrate results for MPSK 
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schemes with throughput 1.0 bps/Hz, 1.5 bps/Hz and 2 0 bps/Hz. For example, the rate 3/4 scheme 
(throughput 1.5 bps/Hz) achieves the same dr {free) (= 8 0) for 4 states, as the rate 6/12 MTCM scheme 
(with set partitioning optimized for the AWGN channel), with lower complexity (multiplicity of symbols: 
(2 vs. 4), number of parallel paths: (2 vs. 8». 

This technique may be useful for transmission on the Rician fading channel with large values of the 
fading parameter, where the requirement for the simultaneous optimization of cnteria for AWGN and 
fadmg channels cannot be achieved by set partitioning of MTCM for fading channels. 


Fractional-Bit Transmission with Single-Symbol TCM 

E. Eleftheriou and P. R. Chevillat IBM Research Division, Zurich Research Laboratory, Sdumerstrasse 
4, CH-8803 Ruschlikon. Switzerland 

Exploiting the bandwidth of a transmission channel to the largest possible extent is often only 
possible if a non-integer number of bits per symbol is sent In this paper we consider the transmission of 
TCM-coded signals with m+plq bits/symbol where q = 2*. We employ single-symbol TCM codes 
together with simple block-coding schemes and signal constellations whose size is not a power of two. 
This combination allows to match the symbol rate to the available channel bandwidth while retaining the 
advantages of single-symbol TCM, e g., a small decoding delay, and the availability of zero-delay 
tentative decisions for decision-directed receiver adaptation In addition, the block code has the property 
that symbols with large amplitudes are sent less frequently improving performance m the presence of 
nonlinear distortion The block-coding operations become particularly simple for p = 1, As an example, 
the transmission of 5.25 bits/symbol is discussed which offers an altcmauvc to the CCITT V.33 scheme 
for 14.4 kb.ts/s modems Finally, the concept of reduced-state combined equalization and trellis decoding 
which mitigates using a larger bandwidth is generalized to the case of fractional-bit transmission. 


Advanced Synchronization Procedures for Trellis Coded MFSK Modems 

B. Ho.iary, F Zolghadr, and M. Darnell Department of Engineering, University of Warwick, Coventry, 

CV4 7AL, UK, and Department of Electronic Eng, University of Hull, Hull, HU6 7RX, UK 

Two of the major problems encountered with communication channels are those of channel errors 
and bit (or symbol) synchronization. Errors are normally countered by the iso of appropriate error control 
coding As a separate consideration the problem of synchromzauon is typically solved by an initial 
synchronization process, followed by either precise subsequent timing at the receiver,, or use of 
segmenting information inserted within the transmission formal Although these procedures are not 
optimum, until recently they have represented the most efficient practical approach possible 

With the advent of powerful and cheap digital signal processing systems, however, it is now 
feasible to unify the two processes, thus potentially producing a more efficient communication system. In 
this presentation an example of such communication system is described The system employs a new 
symbol synchronization method applicable to the reception of trellis coded MFSK signals The procedure 
is termed code-assisted but synchronization (CABS), in which digital processing techniques are used to 
combine the functions of demodulation and error control decoding 

The performance of the CABS modem has teen studied using simulation and practical techniques, 
under addiuve white Gaussian noise channel condition. It is shown that the degradation in performance 
(compared with a perfectly synchronized modem) is less than ldB 


A Trellis Partitioning Technique for Reduced-State Sequence Detection 

Torbjrim Larsson Telecommunication Theory Group, Dept, of Information Theory, Chalmers University 
ofTechnology, S-412 96 Gdteborg, Sweden 

Recently, several related algorithms for reduced-state sequence detccuon have been proposed by 
vanous authors. A joint description of these algorithms can be given by observing that they are all based 
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on a partitioning of the trellis, dividing the set of states into a number (C) of state-classes. The detector 
searches the trellis by saving the K best candidate paths from each state-class Each choice of state-classes 
results in a new search algonthm with an associated minimum distance d ^ This minimum distance 
will, especially for K = 1, limit the detection performance achieved by the algonthm. Thus, it is of 
interest to determine the particular set of state-classes that maximizes 

In this paper a systematic search procedure is employed to find the best (maximal d mm ) partitioning 
of the trellis. As an application, we consider uncoded quadrature amplitude modulation with finite 
intersymbol interference (ISI). For every ISI channel there is a number C* such that if C > C*. then a 
partiuorung with C state-classes can be found with d ma equal to the free distance of the channel For 
minimum phase channels, it is found that C* is usually only a fraction of the number of states in the 
trellis. 


Soft Decision Demodulation and Multi-Dimensional Trellis Coded Phase Modulation 

Joseph M. Nowack and Mark A. Hcrro Department of Electrical and Computer Engineering, University 

of Notre Dame, Notre Dame, IN 46556 

For a multi-dimensional trellis coded phase modulauon system, quantization at the receiver will 
cause some performance degradauon relative to infinitely fine quantisation In this paper, we study a 
variety of soft decision scher...„ that are designed to reduce the effect of quantization at a reasonable cost 
The schemes are compared using cutoff rate calculauons, simulations of multi-dimensional trellis coded 
phase modulation systems, and implementation considerations In addmon, several soft decision schemes 
having central erasure regions are compared based on cutoff rate calculations to study the influence of 
these erasure regions 


Distance Weight Distribution of Trellis Codes Found by DFT 
Torlciv Maseng Elab-Rumt, N-7034 Trondheim, Norway 

A closed form analyuc algebraic formula is given for the distance weight distribution of a trellis 
code This has been done by the application of the transition matrix approach The method uses 
charactcnstic functions in combination with a discrete Founcr transform rather than generating funcuons 
in combination with diffcrcnuauon. This method could be faster to program and could also benefit from 
commonly available special hardware or software in order to do the Founcr transforms 
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A Parallel Systems Approach to Universal Receivers 

Upamanyu Madhow and Michael B Pursley Coordinated Science Laborator , University of Illinois, 
1101 W SpringfieldAve., Urbana,IL61801 

Wc introduce a universal approach to dealing with uncertainty m channel charactcnstics A parallel 
implementation is proposed for a universal receiver that can cope with such uncertainty. The parallel 
implementation consists of a finite number of receivers with the property that, lor any channel in the class 
of interest, the performance of at least one of the receivers will be within a specified degradation of the 
optimal performance The identification of the good receivers is accomplished by means of the intrinsic 
side information generated by an appropnate coding scheme. In this paper, wc give sufficient conditions 
on channel classes for which a universal design as desenbed above is possible. Wc also outline 
procedures for carrying out such a design, and give a general example for A/-ary signaling to illustrate the 
applicability of our theory. Tools for the analysis of the coded performance of our parallel 
implementation arc developed, and it is shown that Rccd-Solomon codes with bounded distance decoding 
satisfy the requirements of a good coding scheme for our application (Research supported by the Army 
Research Office (DAAL-03-87-K-0097)) 


Timing Recovery in the ISDN U-Interface Transceiver 

Erdal Panaytrci Faculty of Electrical and Electronics Engineering. Istanbul Technical University, 
Ayazaga Kampusu, Istanbul, Turkey and TELETAS R&D Department, Umramye, Istanbul, Turkey 

In this paper, a general analysis is presented for tire jitter performance of a common Symbol timing 
Recovery (STR) system employed in a digital subsenber Loop (DSL) transceiver employing adaptive 
echo cancellation for high-speed digital communicauons typical of evolving Integrated Service Digital 
Networks (ISDN's). The STR circuit consists of a prefilter and a squarcr followed by a narrow-band 
postfilter tuned to the signalling rate The output of the timing circuit is a nearby sinusoidal wave whose 
zero crossings indicate the appropriate sampling instants for extraction of the data Exact analytical 
expressions for the mean and vanancc of the timing wave and for the rms phase jitter are denved as a 
function of the bandwidth of the posililter for a given set of input parameters representing a particular 
digital subscriber loop and its noise env.ronmcnl, including such effects as residual-ccho, crosstalk and 
impulse noise Numerical results, obtained for an experimental study of a 144 kbit/s DSL liming recovery 
system show that the presence of these disturbing signals can substantially degrade the synchronizer 
performance. The effect of die excess bandwidth factor of the prefilter on this degradations is also 
mvesugated. 


Some New Results and Interpretations Concerning Binary Orthogonal Signaling Over the Gaussian 
Channel with Unknown Phase/Fading 

Pool Yuen Kam Department of Electrical Engineering. National University of Singapore, Kent Ridge 
0511, Republic of Singapore 

We consider the well-known problem of binary orthogonal signaling over the Gaussian noise 
channel with unknown phase/fading By viewing the problem m a rotated coordinate system, the 
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orthogonal signal structure can be considered as the combination of an antipodal signal set and an 
unmodulated component (pilot tone) for channel measurement. This allows us to show that as far as data 
detection is concerned the optimum matched-filter-envelope-detector is identical to a novel detector- 
esumator receiver in which the detector performs partially coherent detection using an absolute coherent 
reference generated by the estimator from the channel measurement provided by the pilot-tone 
component of the orthogonal signal structure. This detector-estimator interpretation of the optimum 
receive: shows that it is wrong to refer to the latter as a noncoherent receiver. It also leads to the 
development of new approaches for analyzing the error probability performance of the receiver. In 
particular, for the Rician channel, an exponential Chemoff upper bound is obtained, and an expression is 
also obtained for the case of slight fading 


Multidimensional Signaling with Parallel Architectures 

E. Biglicn and F. Pollara Electrical Engineering Department, UCLA, Los Angeles, CA 90024, and Jet 
Propulsion Laboratory, California Institute of Technology, Pasadena, CA 

Imai and Hirakawa and Ginzburg have shown how algebraic codes of increasing Hamming distance 
can be combined with nested signal constellations of decreasing Euclidean distance to generate 
multidimensional signals with large distances. Cusack and Sayegh have shown specific constructions of 
multidimensional constellations. 

Staged demodulation of these multidimensional constellation was also suggested. With this 
procedure the bits of the signal label protected by the most powerful code are decoded first by using 
maximum-likelihood (soft) decoding Then the bits protected by the second most powerful code arc 
decoded, and so on. This procedure is suboptimum, 1 e, the error probability will be increased, but 
reduced decoding complexity wdl result. In particular, staged decoding is amenable to an architecture 
with pipelined parallelism 

In this paper we consider the design of constellations that allow a high degree of parallelism in the 
staged decoder structure, and in addition allow sort decoding of the component algebraic codes by using a 
systolic algorithm suitable for VLSI implementation. 

The resulting receiver structure turns out to be highly modular and flexible, so that different codes 
and different constellations can be accommodated Finally, combination of these multidimensional 
cons'ellations with Trellis-Coded Modulauon (JCM) is considered, and it is shown that this highly 
parallel demodulator structure can work also if a TCM scheme is used to further increase the transmission 
performance. 


Analysis of SCCL As a PN Code Tracking Loop 

Kwang-Chcng Chen and Lee D Davisson Electrical Engineering Department, University of Maryland, 
College Park, Maryland 20742 

Synchronizauon is an essenual issue for PN (pseudo noise) coded spread spectrum communication 
systems. Recent research in this field has concentrated on PN code acquismon. DLL (de'ay-locked loop) 
and TDL (tau-dithcr loop) are still two pnmary types for PN code tracking. Both of the loops are based on 
the design of the early-late gate bit synchronizer. Recently,. Chen and Davisson proposed a new bit 
tracking loop, SCCL. Their design is a digital tracking loop that is based on the Sample-Correlate- 
Choose-Largest procedure. In this paper, we introduce the application of SCCL as a PN code tracking 
loop and analyze its performance both in steady-state and transient conditions. SCCL applying a 
biphasc-level-level signal set acts as a coherent baseband PN code tracking loop which more closely 
implements the MAP optimal structure. Three adjacent estimates are formed by correlating the samples of 
the baseband waveform for each bit We choose the corresponding timing (phase) of the estimate with the 
largest magnitude as the current correct timing (phase) and update it for each bit. Only one summation 
circuit is necessary due to the digital realization of the SCCL The correlation properties of the samples 
from maximum length codes using the biphase-level-level signal set are investigated. A finite-state 
Maikov chain model is used for theoreuca! performance analysis. Its transition probabilities can be 
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represented via triple integrals and calculated numerically The numerical results of performance analysis 
are presented in this paper. 


Adaptive Rate Sampling for Secure Communication Systems 

M. Darnell and B Honary Department of Engineering, University of Warwick, Coventry, CV4 7AL, UK, 
and Department of Electronic Eng., University of Hull, Hull, HU6 7RX, UK 

The paper first considers a procedure for spectral manipulation, based upon a time domain sampling 
algorithm, applied to signals having spectra compnsmg multiple (> 2) separated band-pass elements. The 
algorithm provides an analytical and pracucal tool to enable spectral manipulation to be carried out in 
systematic manner, allowing non-uniform adaptive sampling rates to be applied to a muiuple band-pass 
analog signal if the band structure is first charactenzed Time variauon in the analog signal structure can 
also be tracked. Given that the means are available to identify the form of a spectrum comprising muiuple 
band-pass elements in terms of the transmission frequencies between each of the elements and the 
adjoining spectral ‘gaps', the procedure can be used to drive the minimum sampling rate necessary to 
charactenze that spectrum completely, and hence allow its complete recovery from the samples taken at 
that calculated rate This sampling algorithm is then applied to the scrambling and recombinaUon of a 
muiuple band-pass spectrum. The procedure enables the separated band-pass elements of a frequency- 
scrambled signal, some of which may also be frequency-inverted, to be recombined in their correct order 
and possibly translated in frequency, if required. This would enable the scrambled band-pass elements of,, 
say, a speech signal to be recombined into an intelligible speech signal. It thus provides the basis of a 
speech or data pnvacy/security communication system, in which the secure "key” information comprises 
(a) the spectral frequency ranges of the scrambled elements; and (b) the unique value of sampling rate 
which will allow correct recombination. 

Study of Self-Noise Spectra in Fourth-Power Law Clock Recovery 

Thomas T Fang Lockheed Missiles & Space Company, Inc, Orgn 91-50, Bldg 251, Palo Alto, CA 
94304 

A prior work has established that average jitter power in a clock recovery circuit is a funcuon of the 
self-noise powers in phase with the desired recovered clock and in phase quadrature as well as of the 
cross-spectrum between these two components In this paper, we found a method to compute these 
spectral components for the fourth-power clock recovery technique for the PAM case Comparison is 
made with the squaring type recovery technique using cosine roll-off Nyquist pulses on BPSK mode, at 
several excess bandwidth factors (a). A prefilter is described which completely eliminates umc jitter in 
the fourth-power clock recovery circuit. 


Nonlinear Self-Training Adaptive Equalization for Multilevel Partial-Response Class-IV Systems 
Giovanni Cherubim IBM Research Division, Zurich Research Laboratory, Saumerstr 4, CH-8803 
Ruschlikon, Switzerland 

Self-training adaptive equalizauon for multilevel partial-response class-IV systems is addressed An 
adapuve equalizer realized with distributed-anthmctic architecture is considered, where the process of 
muluplying the tap signals with tap gains and summing the resulting product is replaced by a procedure 
involving only table look-up values and shift-and-add operations. Self-training adaptation schemes 
devised for linear adaptive equalizers do not converge if applied to a distributed-arithmetic equalizer, 
because of the inherent non-lineanty of the system during the adaptauon process In this paper a new 
algorithm allowing the convergence of the look-up values to the optimum setting is proposed and the 
analysis of the dynamics of the look-up values is conducted. Under the usual assumption of independent 
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Numencal results are presented with reference to a multi-level prjv system foMdeh 
transmission over twisted-pair cables. sys!cm ,or ™gb-rate data 


27 


SESSION MP2 


BROADCAST CHANNELS 


Selective Repeat ARQ Schemes for Broadcast Links 

S. Ram Chandran and Shu Lin Department of Electrical Engineering, University of Hawaii at Manoa, 
Honolulu, HI 96822 

In this paper we have proposed several selective repeat ARQ schemes for point to multi-point 
communications, where the receivers have finite buffer size. The schemes proposed here can be classified 
into two categories: the hybrid schemes and the ARQ schemes. The ARQ schemes presented here reduce 
to special cases of the hybrid schemes when the forward error correction part is removed from them. We 
obtain lower bounds on the throughput efficiency of the schemes. Computation of the bounds reveal that 
the schemes perform quite satisfactorily for channels with large round-trip delay and high data rate like 
the satellite channel. All the schemes are adaptive and transmit multiple copies of the message at different 
levels of transmission. The schemes arc simple to implement and outperform the ones proposed by 
Towsley and Mithal and by Mohan, Qian, and Rao. We have also applied the dynamic programming 
approach of Wang and Silvester to our schemes to choose the optimum number of copies The resulting 
schemes give some improvement in throughput. 


Identification for a Deterministic Broadcast Channel 

Bart Verboven and Edward C. van der Meulen Department of Mathematics, Katholieke Umversiteit 
Leuven, Celestijnenlaan 200B, B-3030 Leuven, Belgium 

We denve a direct identification result for the deterministic broadcast channel (BC) We also 
establish a ‘soft converse’ in the sense of Ahlswcdc and Ducck (1989), thereby solving the identification 
problem for this BC. 

Surpnsingly, for the deterministic BC the region of achievable (‘second order') idcnuficaiion rates 
is much larger than the region of achievable transmission rates, in contrast to the one-way channel result 
obtained by Alilswcdc and Ducck More precisely, for the deterministic BC. all the second order rate pairs 
(R\,Rf) satisfying 

OSR, <tf(f,), 

0 <R 2 <,H(Y{), 

where (Y t , Tj) is the channel output corresponding to some input random vanable X, are achievable 
As one can see, in this charactcnzation of the identification capacity region no constraint is needed 
on the sum R t + Ri of both rates, whereas there appears such an extra constraint in the dcscnption of ihe 
transmission capacity region obtained by Pinsker (1978) for this BC. 


Communicating Via a Processing Broadcast Satellite 

F M J Willems, J. K. Wolf, and A. D Wyncr Dept of Electrical Engineering, Eindhoven University of 
Technology, 5600 MB Eindhoven, The Netherlands, Center for Magnetic Recording Research, University 
of California, San Diego, La Jolla, CA 92093, and AT&T Bell Laboratories, Murray Hill, NJ 07974 

Three dependent users are physically separated but communicate with each other via a satellite. 
Each user generates data which it stores locally. In addition, each user sends a message to the satellite 
The satellite processes the messages received from the users and broadcasts one common message to all 
three users. Each user must be capable of reconstructing the data of the other two users based upon the 
broadcast message and us own stored data Our problem is to determine the minimum amount of 
information which must be transmitted to and from the satellite 
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Let the data of the three users at time n, (1 £» <~0 be (X„ Y„Z .), which are statistically 
independent drawings of the dependent vector (X, Y, Z) Let R x , R y , and R x be the rates of transmission 
up to the satellite by the three users, and let To be the rate of transmission down from the satellite. We 
show that for data blocks of size N, where N is large, the set of rates (R x , R y , R,, R g ) which satisfy 

R x >H(X\Y,Z), R y >H(Y\X,Z), R x >H(Z,X,Y), 

R x +R y >H(X, Y\Z), R y +R,>H(Y,Z\X), R S +R,>H(X,Z\Y), 

R 0 > max[H(X, Y |Z), H(X,Z | Y), H(Y, Z |X)} 

is necessary and sufficient for the purpose outlined above 


Some Matching Results in Multi User Communication 

Sergei I. Gelfand and Edward C van der Meulen Institute for Problems of Information Transmission, 
USSR Academy of Sciences, 19, Ermolova Street, 101447, GSP-4 Moscow, USSR, and Department of 
Mathematics, Kathoheke Umversiteit Leuven, Celestijnenlaan 200 B, B-3030 Leuven, Belgium 

We derive necessary and sufficient conditions for the reliable transmission of a two-component 
correlated source over a multi-user channel in two situations First, matching condiuons arc derived for 
sending an arbitranly correlated two-component source with arbitrarily small probability of error over a 
capability-degraded broadcast channel (BC) A discrete memoryless BC [X.Ply.z |x), Y xX ) is said to 
be capability-degraded if l(X;Z)<I(X\Y) for ail probability distributions P(x) on X. These matching 
conditions are stated in a computable form Secondly, we show dial the sufficient condiuons, obtained by 
Cover, El Gamal, and Salehi (1980) for reliable transmission of a two-component correlated source over a 
discrete memorylcss multiple-access channel (MAC), are also necessary, if the source (S,T) satisfies the 
condiuon that S and T arc conditionally independent given their common pan K. The matching conditions 
obtained this way include several previously known cases for which necessary and sufficient conditions 
were found for sending a correlated source reliably over a MAC. 


Suboptimal Link Scheduling in a Network of Directed Transceivers 

Galen Sasaki Department of Electrical and Computer Engineering, The University of Texas, Austin, TX 
7S712-10S4 

The problem of scheduling data transfers in a network ( V,E ) of transceivers, where preemption of 
transfers is allowed and transmissions arc directed, has been studied by Choi and Hakimi among others 
In the network each node v has b(v) transceivers, each link e can have at most c(e) simultaneously 
communicating pairs of transceivers, and L, is the size of the smallest odd cycle of (V,E) (hence, L,> 3) 
Choi and Hakimi provided sub-optimal scheduling algorithms that have time complexity 
0 (| £ | 1 y b (v)) and produce schedules of length at most (1 + —] z where r is the length of the 

optimal schedule For the case when c([u,v))e {0, 1) (resp, c((«,vj)6 (0, min (b(v), b(v)))) for all 
links [«,v], we provide an 0(|£ | 2 1 V |) (resp., 0(\E | |K p)) time algomhm that produces a schedule 
with length at most 

{1 + 2f(L r +l)min yt i/*(v) +Z-,-3)' 1 )r (resp , {1 + (L, mm yc ^h(v)-/ [min ye vh(v)is odd]]' 1 K) 

For the case when c(e)e (0, I) for all links e, the scheduling algorithm can be modified to output the 
schedule in certain useful representations and have smaller time complexities (0(|E||F| ? ) or 0(|£| 2 ) 
depending on the representauon) 
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Accessing an Unbounded User Population by Decimation Codes, Using Feedback Only Once Per 
Newcomer 

Sindor Csibi Technical University of Budapest. Stociek utca 2, H-UII, Budapest, Hungary 

Accessing simultaneously at most K members of an unbounded population of nonregistered 
potential users is considered via a collision channel, with a probability 1 -p, by means of decimation 
codes. Let us reserve L protocol sequences for this purpose from a binary protocol sequence set of size T 
and a block length N. Let us leave only the rest, viz., T-L sequences for the identification of registered 
users (which still may be either active or just potential). Feedback has to be used for so doing but only 
once per nonregistered newcomer, just to accomplish first ntrance Reed-Solomon code techniques well 
known for accessing an entirely registered potential user population of a size at most S without feedback 
are extended, taking into account the new iask of facing a populanon of potential users of not a given but 
an unbounded size, and exploiting further the rich possibilities of the structure. 


Uniquely Decodable Codes for the Two-User Binary Adder Channel 

Feng Guo and Yoichiro Watanabe Department of Electronics, Faculty of Engineering, Doshisha 
University, Kyoto, 602 Japan 

A uniquely decodable (UD) block-code pair (Ci,C 2 ) for the two-user binary adder channel (BAC) 
is constructed under the condition that the code Ci is linear In 1983, Kasami et al showed that a code 
pair (C,,C 2 ) for the two-user BAC is UD if the code C 2 is an independent set of the graph Gc x (V,E) 
associated with the code C t A problem is how to pick out a linear code C i with a given rate R } , in order 
to maximize the rate fij of Cj which is a maximum independent set of the graph G C] (V,E). The 
proposed UD code pair is constructive and its rates are given by 

1 

t + r'' 

— log 2 (2 M -1)+ — + Iog 2 (2' -1) 

/ + r t-r r 

Here Ci is an (n,k) linear code with code length n = tk + b and r = blk, where t, b are integers and 
t £ 1, 0<b < k. For 0.5 </?> S l(r = 1), the rate R 2 reduces to (1 —/?i)log 2 3, which achieves the upper 
bound derived by Kasami-Un and Weldon in 1978 In other words, when a ratc /? j of the linear code C \ 
is given as 0.5 < R i £ 1 a prion, a UD code pair (Ci t c,) in which S, takes the maximum can be produced 
without umc shanng. 


*1 = 
/? 2 = 


Connections Between Exponential Sums, Algebraic Curves and Sequence Design 
P Vijay Kumar and Oscar Moreno EE-Systems, University of Southern California, Los Angeles, CA 
90089-0272 and University of Puerto Rico 

An application of Deligne’s bound on Weil (exponenuai) sums in several variables is shown to 
yield an asymptotically opumum family of p-ary sequences (p an odd prime) whose performance is 
superior to that of either the Gold (by 3 dB) or the Kasami family of binary sequences. Thus, a strong 
argument for the use of nonbinary sequences for CDMA is made. It is shown that all sequences in the 
family except one are bent functions. The distribution of correlation values is also evaluated. 

A comparison with other previous nonbinary designs known to the authors shows the present design 
to outperform all others included in the comparison The new sequences are easily implemented using 
shift-registers, especially since the size of the sequence alphabet (p ) can be made as small as 3 Two other 
designs are also presented. 

On a related subject, it is shown how the recent results of Lachaud and Wolfmann on Kloosterman 
sums and their application to coding theory can be reinterpreted so as to settle a conjecture concerning the 
inner product of an m-sequence with its reciprocal. 
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ESTIMATION I 


On Universally Efficient Parameter Estimation in Parametric Models and Universal Data 
Compression 

N. Mertiav and J. Ziv AT&T Bell Laboratories, Murray Hill, NJ 07974, and Department of Electrical 
Engineering, Technion, Haifa 32000, Israel 

We consider estimation in paramedic models such as the location mode) and the autoregressive 
model for the case where the probability density function of the noise is completely unknown We say 
that an estimator is univeisally asymptotically efficient if it does not depend on the unknown noise 
density function and at the same time attains the Crame'r-Rao lower bound uniformly for any well- 
behaved density. For these statistical models, universal asymptotically almost efficient estimators are 
proposed under certain regularity conditions. These estimators are strongly related to universal data 
compression and to universal hypothesis testing algonthms m a unified framework. 


Estimating the Number of States of a Finite-State Source 

Jacob Ziv and Neri Mertiav Department of Electrical Engineering, Technion -U T, Haifa 32000, Israel, 
and Speech Research Department, AT&T Bell Laboratories, Murray Hill, NJ 07974 

We study the problem of estimaung the number of states m a finite-state, finite-alphabet source. The 
following performance entenon is adopted: minimize the probability of estimating the number of states 
while keeping the overcstimation probability exponent at a given presenbed level. 

A universal asymptotically optimal estimator, in the sense just defined, is proposed first, for the 
general class of finite state sources, and later for several important subclasses The optimal estimator 
proposed relies on the Lempcl-Ziv (LZ) data compression algorithm in an intuitively appealing manner. 


A Geometric Interpretation of the Linear Set-valued Estimator 

Darryl R. Morrell and Wynn C. Stirring Electrical and Computer Engineering Department, Arizona State 
University, Tempe, AZ 85287-5706, and Electrical and Computer Engineering Department, 459 Clyde 
Building, Brigham Young University, Provo, Utah 84602 

Recently, a theory of discrete-time opumal estimation (filtenng, smoothing, and prediction) based 
on convex sets of probability distnbunons has been developed. By rcstneting attention to the linear 
Gaussian problem, a set-valued estimator is obtained; the estimator is an exact soluuon to the problem of 
running an infinity of Kalman filters (and fixed-interval smoothers), each with different initial condiuons. 
In this paper, the philosophical basis underlying the theory of set-valued estimauon is presented and the 
estimator developed for the linear Gaussian problem is bnefiy reviewed. A geometrical interpretation of 
this estimator is presented: this interprctauon provides a natural and informauve framework in which the 
set-valued estimator can be understood In addmon, the geometnc interprctauon leads to a significant 
generalization in the sets that can be represented in the set-valued estimation algorithms. 


Further Results on Resistance in Detection and Estimation 

Kenneth S. Vastola Electrical, Computer, and Systems Engineering Department, Rensselaer Polytechnic 
Institute, Troy, NY 12180 

Resistance of detection and estimation procedures is considered In previous work we showed that, 
for procedures formed by summing the output of a finite memory non-linearity, a simple condition exists 
for uniform resistance. 
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In this talk we will consider other statistics. For example, we will show that the median is not 
unifomtly resistant, no are finite length medians (i e„ median filters). However, conditions for strong 
resistance have been developed by Boente, Fraiman and Yohai for this case. We will use these and other 
examples to compare the data oriented uniform resistance approach with the probabilistic strong 
resistance approach. 


Binary Input Sequences for Maximum-Likelihood Estimation of Multipath Channels 

J. Ruprecht and J L. Massey Institute for Signal and Information Processing, ETH, CH 8092 Zurich, 

Switzerland 

A discrete-time multipath channel estimation system in the presence of additive white Gaussian 
noise is considered A sequence s is sent through the channel, which is modeled by a linear system whose 
unit sample response is unknown The problem is to esumate the terms in the unit-sample response 
according to the maximum-likelihood (ML) esumation rule. 

If the unit sample response is known to be non-zero only over a known range, a periodic sequence s 
may be used as the channel input. If nothing is known in advance about the unit sample response, an 
apenodic sequence s with a finite number of non-zero digits is the appropnatc input sequence In both 
cases, the inverse filter for the sequence s is shown to be the ML estimator Critena for an optimum 
sequence choice are derived For binary sequences, which are the most important in applicauons, the 
optimization entena reduce to maximizing the processing gam of the inverse filter, where the processing 
gain is defined as the ratio of 'he signal-tc-noise rauos at the output and the input, respectively, of the 
estimator 

Optimum binary sequences up to penod or length 45, respectively, arc presented. For these 
optimum sequences j, there is a loss of less than 1 dB of processing gam compared to the matched filter, 
wluch is known to be the estimator that maximizes the processing gain. The inverse filter, however, 
eliminates completely the sidclobcs that appear in matched filter processing and that cause the matched 
filter to be a poor channel estimator for a multipath channel. The realization of the inverse filter as a 
stable filter for the apenodic sequence case is considered, and a practical close approximauon to the true 
inverse filter is derived 


On the Delay Estimation of Discontinuous Signals 

K. Kosbar and A. Polydoros Department of Electrical Engineering, University of Missouri-Rolla, Rolla, 
MO 65401, and Department of Electrical Engineering/Systems, Communication Sciences Institute. PHE 
414, University of Southern California, Los Angeles, CA 90089 

A new lower bound is developed for the vanancc of Maximum-Likelihood (ML) time-delay 
esumauon when the received signal is a square pulse, corrupted by additive white Gaussian noise. This 
models a vanety of practical situauons of interest in communications and radar, whenever the employed 
signal is wideband The bound is generated by combining concepts previously developed for a special 
class of stochastic processes, induced by the signal model For moderate signal to noise rauos, the new 
bound is significantly tighter than previously known ones As an appheauon example, it is used here to 
suggest an optimal pulse width that minimizes the vanancc of the delay-estimation error 


Constrained Distributed Estimation and Quantization for Distributed Estimation Systems 

John A Gubncr Department of Electrical and Computer Engineering, University ofWisconsin-Madison, 
Madison. Wl 53706-1691 

Consider a distributed estimation system consisting of n sensor platforms taking respecuve 
measurements, F|, ,Y, After local processing, each sensor transmits its findings over a communication 
channel to a common fusion center the purpose of this system is to estimate some unobservable random 
variable, X. We assume that the sensors do not communicate with each other,, and that there is no 
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feedback from the fusion center to the sensor platforms. In the appropriate probabilistic setting, it is well 
known that the minimum-mean-square-error estimate of X given Y t , ,.,Y n is E [X \ Fj, .,y„]. However, 
usmg any physical communication system, it is not possible to transmit real-valued quantities without 
distortion. Hence, it is not possible for the fusion center to compute E [X lY^ In order to 
formulate a system model in such a way that we have some control over the distortion of the data 
transmuted to the fusion center, we shall consider the simultaneous optimization of the choice of a 
constrained fusion rule together with the choice of quantizers for the sensor platforms 


Recursive Pseudo Maximum-Likelihood Estimation for Joint Carrier Phase and Symbol Timing 
Recovery 

Yih-Fu Won and Chung-Chin Lu National Tsing-Hwa University, Institute of Electrical Engineering, 
Hsin-Chu, Taiwan, 30043 R.O C 

A recursive on-line algonthm for joint carrier phase and symbol timing recovery of lmear- 
modulated systems, called recursive pseudo maximum-likelihood estimation (RPMLE), has been derived 
based on the maximization of a negative cost function consisting of pseudo likelihood observations. 
RPMLE uses the inverse of accumulated Hessian matrices of pseudo likelihood functions as weighting 
matrices to update timing parameters recursively. RPMLE can be applied to baseband PAM timing 
recovery by a similar approach. Several synchronizers are presented, which accommodate either data- 
aided or nondata-aided estimation according to whether detected data is directed into the estimators 
Meanwhile, we find that the hacking loop implementation for joint earner phase and symbol timing 
recovery developed by Meyers and Franks is a special case of RPMLE with constant weighting marnx 
Simulation results show that the rate of convergence of RPMLE is faster than that of stochastic 
approximation. RPMLE owns high noise immunity and can be applied to solving synchronization 
problems of high transmission rate systems 
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SESSION MP4 


QUANTIZATION I 


Adaptive Entropy-Coded Predictive Vector Quantization of Images 

J W. Modestino and Y. H. Kim Electrical, Computer and Systems Engineering Department, Rensselaer 
Polytechnic Institute, Troy, NT I2I80 (40 min.) 

We describe a new approach to image coding based upon adaptive entropy-coded 2-D predictive 
vector quantization (PVQ) ideas. PVQ is a straightforward vector extension of ordinary scaler predictive 
quantization schemes, such as DPCM, where the vector quantizer (VQ) is now embedded in the 
predictive feedback loop. Prediction is then performed on a vector, or block, basis using previously 
encoded blocks with the prediction error blocks subsequently applied, on a block-by-block basis, to me 
VQ. While PVQ is not new, previous applications have not attempted to exploit the further 
compressibility of the VQ output through use of vanabie-length entropy coding. In this paper we consider 
2-D PVQ of images subject to an entropy constraint and demonstrate the substantial performance 
improvements over existing approaches. Furthermore, we desenbe a simple adaptive buffer-instrumented 
implementation of this 2-D entropy-coded PVQ scheme which can accommodate the associated vanabie- 
length entropy coding while completely eliminating buffer overflow/underflow problems at the expense of 
only a slight degradation in performance. This scheme, called 2-D PVQ/AECQ, is shown to result in 
excellent rate-distortion performance and impressive quality reconstructions on real-world images. 
Indeed, the real-world coding results shown here are rather sinking and demonstrate almost imperceptible 
distortions at rates as low as 0.5 bits/pixel 


Necessary Conditions for the Optimality of Residual Vector Quantizers 

Chnstophcr F. Bamcs and Richard L Frost Department of Electrical & Computer Engineering, Brigham 
Young University, Provo, Utah 84604 

Multistage or residual vector quantizers (R VQs) previously have been designed such that each stage 
satisfies conditions necessary for the optimality of single stage exhaustive search vector quantizers 
Consequently, each stage of the RVQ is optimized independently of all other stages This independent 
stagcwisc optimization causes performance to suffer whenever two or more stages arc concatenated. In 
this paper the structure of residual quantizers is considered and the structural implications of these 
quantizers arc made explicit for the RVQ encoder and decoder Once the structural constraints arc made 
explicit, we are able to determine necessary conditions for the joint optimality of all stages We first 
determine necessary conditions for minimum mean square error lor the quanta and partiuons of residual 
scalar quantizers. The dcnvation of the scaler results mouvatc an approach yelding a vector 
generalization of the opumality conditions which hold for a broad class of distortion measures. An 
algonthm is given which yields RVQs which satisfy these necessary condmons and performance 
comptuisons arc given between conventionally designed RVQs and RVQs designed with the new 
algonthm 


Quantization for Decentralized Estimation from Correlated Data 

M. Di Bisceghc and M Longo SelemaSpA , Radar Dept, Via Ttburtina, km 12,400,1-00131 Roma, 
Italy, and Untversita'di Napoli, Dipartimento di Ingegneria Elettronica, Via Claudio 21,1-80125 Napoli, 
Italy 

In a decentralized estimation scheme with two sensors data are remotely encoded by scalar 
quantizer to fulfill capacity constraints of channels conveying information to a central estimator. The 
quantizers are to operate separately, but a joint model of observations - allowing for spatially correlated 
data - may be taken into account at live design stage. We address the following problem 
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Given: the source distribution P x , the observation model Py„y,< a distortion measure d(x, x); find 
quantizers Q\, Qi : Y t x Y 2 —» M i x M 2 with rates R\ R-i, and estimator Z=g(m t m 2 ), nti,, 
m 2 e M 1 xM 2 ; such that the average distortion D = Ed(X, X) is mmimum 
We prove that, for given Q\Qi a nd g, the sequence 
1- 2t(yi) = argmin„ l6Ml J x d( - x ’ • t "(mi.'« 2 »Pfc m 2 ,y\)dx 
2. g + (m 1 , m 2 ) = argmin.,- d(x, x)p(x \m u m 2 )dx 

is a descent step forD. By alternately applying this iteration to both quantizers a design algonthm results 
For a linear Gaussian observation model we derive the limit performance of any encoding-decoding 
scheme, m terms of an upper bound to the rate-distortion function GqIP), where Rc = #1 + /? 2 - Under 
various combinations of per-channel signal-to-noise ratios and spatial correlation coefficients, quantizers 
designed via the proposed algorithm nearly achieve such limit performance. 


Optimal Quantization and Fusion in Multiple Sensor Systems with Correlated Observations 
Yawgeng A. Chau and Hvaggelos Geraniotis Department of Electrical Engineering <4 Systems Research 
Center, University of Maryland, College Park, MD 20742 

In this paper we address two problems of quantization am. Jala fusion from multiple sensors for the 
detection of a weak signal in dependent noise, ( 1 ) fusion from sensors with mutually independent 
observations and ( 11 ) fusion from sensors with correlated observations. In the first problem, the 
observations of each sensor consist of a common weak signal disturbed by an additive stationary in¬ 
dependent, p-mixing, or p -mixing noise process. The noise processes of the individual sensors are 
mutually independent. In the second problem the noise processes of the different sensors arc correlated. 

Three distinct schemes involving (a) fusing the test statistics formed by the sensors without 
previous quantization, (b) quantizing directly each of the sensor observations and then fusing, and (c) 
quantizing the test statistics of the sensors and then fusing them are considered The memoryless 
nonlincantics, as well as the break-points and quantization levels of the quantizers introduced in these 
quantmation/fusion schemes are obtained as solutions to appropriate integral equations which result from 
the optimization of suitable measures (error probabilities and deflection) of the performance of the fusion 
center 


Joint Vector Quantizer and Signal Constellation Design for the Gaussian Channel 

Michael G Perkins The German Aerospace Research Establishment (DLR), NE-NT-T, Oberpfaffenhofen, 

8031 Wessling/Obb , West Germany 

The usual approach to combined sourcc/channcl coding is to design the source and channel coders 
independently, however, in many cases significant gains can be achieved by jointly designing the two 
coders This paper addresses the problem of joint vector quantizer and signal constellation design for the 
Gaussian channel. It is assumed that the transmitter is operating under a peak power constraint, that 
coherent modulation with perfect transmittcr/reccivcr earner-phase synchronization is employed, and that 
the receiver makes maximum apostenori decisions. Under these assumptions, an algonthm for finding a 
locally optimal vcctor-quantizcr/signal-constcllation pair is presented. 

The algonthm makes use of two sub-algonthms. one for optimizing the design of a vector quantizer 
for a given signal constellation, and one for optimizing the design of a signal constellation for a given 
vector quantizer. Starting with an initial vector quantizer and signal constellation, these two sub¬ 
algorithms are repeatedly applied, first one then the other, until neither the vector quantizer nor the signal 
constellation changes significantly from its previous state. 

Simulation results for a low-rate image coder communicating over a noisy channel arc presented. 
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Bennett’s Integral for Vector Quantizers, and Applications 

Sangsin Na and David L. Neuhoff Dept of Electrical Engineering, University of Nebraska - Lincoln, 
Lincoln, NE, 68588, and Department of EECS, The University of Michigan, Ann Arbor, Ml 48109 

This paper extends Bennett’s integral to vector quantizers and «h power distortion measures. The 
result is a simple foimula that expresses the distortion of a many-point vector quantizer in terms of the 
source density, the number and density of quantization points, and the inertial density of the quantizer. 
The latter is a (unction that, at a given point x, approxii ately equals the normalized moment of inertia of 
the quantization cells around x. Previous extensions have not included the inertial density, which limited 
their applicability. The new version is formulated in terms of a sequence of quantizers whose point and 
inertial densities approach known funcuons. Precise conditions are given for the convergence of the 
distortion to Bennett's integral. A Bennett-like approximate formula for the entropy of the quantizer 
output is also developed. 

Applicauon of the new Bennett's integral leads to- a rigorous derivation of the distortion of multi¬ 
dimensional companders; the conclusion that for memoryless sources, the advantage of vector quantizers 
over scalar lies principally in their superior inertial densities, rather than point densities; and a 
quanutanve analysis of the increase in distortion of various structured vector quantizers (e g„ tree- 
structured or block-transform) due to their formation of quantization cells with relatively few faces. 


A New Algorithm for the Design of Locally Optimal Adaptive Vector Quantizers (AVQ) 

G Szekeres and G. Gabor Information Theory Group, Hungarian Academy of Sciences, Budapest , 
Hungary, H-l 111, and Dept, of Math Stats & Comp Sci .Dalhousie University, Halifax, NS Canada 
After specifying an appropriate notion of optimality, three necessary conditions of optimality are 
established for forward adaptive VQ’s, two of which are analogous to that of Lloyd and Gray et al and 
the third is related to the adaptive nature of the VQ Based on these conditions a convergent iterative 
algonthm is developed which uses cither a long training sequence or the true distnbuuon for the design of 
adaptivity (next-state function) and a set of VQ’s with nonincreasing distortion in each step The same 
algorithm is also used as a feed-forward design for backward adaptive VQ's Experimental results with 
speech waveforms arc also discussed 


Finite-State Vector Quantizers for Channel-Error-Resistance 

Lei Ye and Zheng Hu University of Electronic Science and Technology of China, Chengdu, PR China 
and Xidian University, Xian, PJ1 of China 

Since a finite-state vector quantizer is a variety of tracking finite-state systems, channel errors can 
have disastrous effects The degree of the effects and means to combat it are very important in theory and 
practice. In this paper, we study finuc-statc vector quantization of channei-error-resistance in noisy 
channels The sufficient condition for error-finite propagation and the existence theorems for finite-state 
vector quantizers with bounded-error propagation arc proved A formula for estimating performance of 
error propagation in finite-state vector quantizers of channel error-resistance is derived 
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SESSION MP5 


SHANNON THEORY I 


A New Outlook on Shannon’s Information Measures 

Raymond W. Yeung AT&T Bell Laboratories, Crawfords Corner, Holmdel, NJ 07733-1988 

We formalize the previous work of Reza, and Csiszar and Komer on the underlying mathematical 

structure of Shannon’s Information measures. Let X it i = 1. n be discrete random variables. By 

regarding random variables as set variables, let 0 = uX, be the universal set and F be the cr-field 

I 

generated by {X,}. We show that Shannon’s information measures constitute a unique measure on F. To 
be precise, the Shannon information measure (i.e., Shannon’s information measures as a whole) is a 
measure on F. This point of view, which we believe is of fundamental importance, has apparently been 
overlooked in the past by information theorists. As an immediate consequence we introduce the I- 
Diagram, which is a geometrical representation of the relationship among the information measures. The 
I-Diagram is similar to the Venn Diagram in set theory. We discuss the use of the I-diagram; some 
applications of which reveal previously unknown results. We also propose the mutual information 
measure for an arbitrary number of random variables and discuss some of its properties. 


Finding a Basis for the Characteristic Ideal of an /{-Dimensional Linear Recurring Sequence 

Patrick Fitzpatrick and Graham Norton University College, Cork, Ireland and University of Bristol, 
United Kingdom 

Let IV denote the set {0,1,...}, and IV" the cartesian product of n copies of IN, and let IF be a field. 

We denote the power series ring F[(X j.X„]] by ^[[Xl) and abbreviate the monomial X\' • • • X 1 ^ to 

X 1 for i e IV". Let (cr) := (ct|) be a sequence of elements from IF indexed by IV". 

If (cr) satisfies a linear recurrence relation of the form 

X /* CT *+< = 0 for all i > 0 

seS 

where 5 is some finite non-empty subset of IN n and /, e F for all s, then (cr) is called an n-dimensional 
linear recurring sequence (or n-D Irs ) in F. The corresponding polynomial 

/(X): = Z/,X* 

S€ S 

in F[X] is the characteristic polynomial of (a) associated with the relation above. For convenience we 
define the zero polynomial to be a characteristic polynomial of every n-D Irs. 

It is easy to see that the set /(cr) of characteristic polynomials of (cr) forms an ideal in F[X]: this is 
the characteristic idea of (cr). By Hilbert’s Basis Theorem, /(cr) has a finite set of generators. Our aim in 
this paper is to describe a constructive method (Algorithm IDEALBASE) by which such a basis may be 
determined, in the case that (cr) is rectilinear, that is, when f(cr) contains a polynomial Pk(X k ) with 
p*(0) *0 for each k = l,...,/i. It will be observed that this assumption is reasonable, in view of the fact 
that our methods apply in particular to doubly periodic arrays, 2-dimensional cyclic (or TDQ codes, and 
also to more general polynomial codes in several variables. Moreover, our results may be applied to 
related areas such as the theory of the rational transfer functions associated with the synthesis of digital 
filters. 
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The Shannon-McMillan-Breiman Theorem and Other Information Theory Results Via a New 
Ergodic Theorem, 

John C. Kieffer Department of Electrical Engineering, University of Minnesota, 200 Union Street, S E, 
Minneapolis, Minnesota 55455 

Let A be a finite set and let Xi,X 2 , ■ • be a stationary process with state space A For each positive 
integer n, let F„ be a nonempty family of real-valued functions on A". A new ergodic theorem is 
presented, which, if certain assumptions about the sequence of families (F„} are sansfied, allows one to 
say the following: (1) there is an essentially unique function ' e F„ for which E f,(X\, ■ ■ ,X„) is 
minimized as/„ ranges over F„; and (2) the random variable f, (X i, • • • ,X„)/n converges almost surely 
as n->°° A sketch of the proof of this theorem is given. Also, a survey is given of some of the 
information theory results (including the Shannon-McMillan-Breiman Theorem) that are obtainable via 
the new ergodic theorem. (Research supported by NSF Grant NCR-8702176.) 


Shannon’s Coding Strategies for the Two-Way Channel-A Computer Attack 
J Pieter M. Schaikwijk Department of Electrical Engineering, Eindhoven University of Technology, Den 
Dolech 2, P 0 Box 513,560 MB, Eindhoven, The Netherlands 

Shannon derived an expression for the capacity region of the two-way channel (TWC) as the 
limiting rate, normalized with respect to the block length n, of optimum fixed length coding strategies of 
increasing block length n. These coding strategics can conveniently be represented as strategies for 
subdividing the umt square We will report on the results of a computer study of these subdivisions of the 
unit square The computer can yield effective strategies by simulated annealing. For very short (n = 3) 
strategies is it possible to find the best mixed strategy by exhausuveiy testing all possible sets of pure (i e. 
Rows and columns of the unit square) strategies The computer study is undertaken m a renewed effort to 
tightly upper bound the rate of certain deterministic TWC's, such as Blackwell’s binary multiplying 
channel (BMC). We conclude by considenng the prospects of achieving our ultimate goal 


A Sperner-Type Theorem and "Symmetric Versions" of Zero-Error Capacities 
J Komcr and G. Simonyi Mathematical Institute of the Hungarian Academy of Sciences, H-1364 
Budapest, P 0. Box 127, Hungary 

Several classical combmalonal problems ansc through a certain symmctrizatlon of the concept of 
zero-error capacity of compound discrete memorylcss channels. We shall show that in some cases the 
more severe rcstnctions so imposed do not effect the asymptotic results. We prove in particular that if 
N(n) denotes the largest cardinality of a family (0|, D 2 , ,£>„) of subsets of a set of n elements such 
that for every i the set D, is the disjoint union ot an A, and B, with 

A, cf D t , B, cj: D t for every ] * i, 
then [/Vfn)! 1 '" converges to (1 + \ r 5)l 2 


Information Rates of Subsets and Matrix Inequalities 

Thomas Cover and Joy Thomas Departments of Electrical Engineering and Statistics, Stanford 
University 

The entropy per element of a randomly chosen subset of a set of random variables decreases with 
set size, thus proving Szasz’s suing of determinant incqualiucs. We show that the conditional entropy per 
element increases with set size, thereby proving a similar set of determinant inequalities. Yet another 
stnng of determinant inequalities follow from the monotonicity of mutual information between subsets of 
random variables. 
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Feedback in Discrete Communication 

Alon Orlitsky AT&T Beil Laboratories, 600 Mountain Avenue, Murray Hill, NJ 07974-2070 (40 min.) 

X and _y are finite sets (X, Y) is a random variable distribn' d over Xxy according to some 
probability distribution p(x,y). Person P x knows X, Person P y knows Y, and both know p. They 
communicate in order for P y to learn X. P x may or may not leam /. How many information bus must be 
transmitted (by both persons) m the worst case? 

Ci(p) is the number of bits required when only one message is allowed, necessanly from P x to P y 
Clip) is the number of bits required when only two messages are permitted. P y transmits a message to P x 
then P x responds with a message to P y . C„(p) is the number of bits required when P x and P y can 
commumcate back and forth. Messages from P y to P x are called feedback. 

The maximal reduction in communication achievable via feedback is one bit away from 
logarithmic: all probability distributions p have C_(p)i[logC ;</>)] + 1 while there are distnbuuons for 
which equality holds, therefore C\ip) can be exponentially larger than C_(p). Yet C 2 <P) cannot With 
just one feedback message the number of bits required is at most four times larger than the minimal: 
C 2 (p)<4C„0) + 2. 

Surpnsingly, for almost all sparse probability distnbuuons, P y who wants to say nothing must 
transmit almost all the bits in order to achieve the minimal number: C_(p) The number of bits 
transmitted by P y can be appreciably reduced only if P x transmits exponentially more than C„(p) bits 

If the communicators can tolerate e probability of P y not knowing the correct value of X and if 
randomized protocols are allowed then4C»(p) + 21og(l/e) bits suffice even without feedback. 
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CODING THEORY H 


Decoding is Really Hard 

Jehoshua Brack and Morn Naor IBM Almaden Research Center, 650 Harry Road, San Jose, CA 95120- 
6099 

The problem of Maximum Likelihood Decoding of linear block codes is known to be hard We 
show that the problem remains hard even if the code is known in advance and can be preproccssed for as 
long as desired in order to devise a decoding algorithm. The hardness is based on that an existence of a 
polynomial nme algorithm implies that the polynomial hierarchy collapses Namely, some linear block 
codes probably do not have an efficient decoder The proof is based on results m complexity theory that 
relate uniform and nonumfomi complexity classes. 


A Generalization of the Discrete Fourier Transform in Finite Fields 

Peter Mathys Department of ECE, Box 425, University of Colorado, Boulder, CO 80309 

Let v denote a vector of length N over a finite field of characteristic p Then a spectral representation 
V of v can be obtained by using the discrete Fourier transform (DFT) in a finite extension field of 
charactcnsuc p, provided that p and N are rclauvcly pnme It is shown that the DFT in finite fields can be 
generalized quite naturally to include block-lengths N wmch are divisible by p The resulhng spectral 
description of vectors over finite fields appears to be useful for the charactenzation of certain block codes 
For the special case where N is a power of p, a simple charactenzation of (generalized) Rced-Muller 
codes can be obtained which is equivalent to the one given by Massey, Costello, and Justcscn 


Upper and Lower Bounds on Aliasing Probability of Some Signature Analysis Registers 

Torn Fujiwara, Feng Shou-ping, and Tadao Kasami Department of Information and Computer Sciences, 

Faculty of Engineering Science, Osaka University, Toyonaka, Osaka 560 Japan 

In sclf-tesung of LSI, a single or multiple input linear feedback shift register (LFSR) is widely used 
as a signature analysis register An LFSR can be considered to be a circuit for dividing an input 
polynomial by a polynomial g(x) It is known that the probability of an aliasing error in the LFSR is 
equal to that of an undetected error of a shortened cyclic code generated by g(X) when the code is used 
for error detection Here, we assume that the probability of an error occurring at an output bit of the 
circuit under test is a constant e 

For a binary code C, let P,(C,e) be the probability of undetected error of C for a binary symmetne 
channel with bit-error rate e Let HM' denote a shortened ( n,n-r) Hamming code, and RSH(g(X)) 
denote a binary cede obtained from a shortened Rccd-Solomon code over GF(2 m ) of length n (symbols) 
whose generator polynomial is g(X) We present several upper and lower bounds on P,(C,e) for H M* 
and R S?(g (X)) with g (X) = (X-a), (X - 1)(X - a), (X-a KX - or 2 ). or (X -1 )(.Y -a )(X - or 2 ) 

We show that 

0 9 x 2~ 32 £ P'(HM?, 0 2) £ 1.1 x 2' 32 , for 303 £ n £ 2 32 -1 
and 

0 9 X 2 32 £ P,(HM 32 , 0 05) £ I 1 x 2' 32 , for 1210 £ n £ 2 32 -1 

We also show that 

0 99 x 2‘ 32 £ P,(R Sl 2 (X-a), 0 2) £ 1 01 x 2' 32 , for 32 £ n £ 2 32 - 1, 
and 

0 9 x 2~ 32 £ P,(R S 32 (X -a), 0 05) £ 1 1 x 2' 32 , for 63 £ n £ 2 32 - 1 
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A Strengthening of the Assmus-Mattson Theorem 

A. R. Calderbank, P. Delsaite, and N. J A. Sloane Mathematical Sciences Research Center, AT&T Bell 
Laboratories, Murray Hill, NJ 07974 Philips Research Laboratories, Avenue van Becelaere, B-1170 
Brussels, Belgium; and Mathematical Sciences Research Center, AT&T Bell Laboratories, Murray Hill, 
NJ 07974 

Let tvi =d, v/j,. , w, be the weights of the non-zero codewords in a binary linear [n, k, d] code C., 
and let w iW 2 ',, w/, be the non-zero weights in the dual code C L . Let t be an integer in the range 
0<r cdsuch that there are at most d-t weights w,' with 0<w/S«-t. Assmus and Mattson proved that 
the words of any weight w, in C form a r-design Let 8 =0 or 1, according as C is even (w/ = n) or not, 
and let B denote the set of codewords of weight d We prove that if W2>d+4, then either (1) t = 1, d is 
odd, and B partitions {1,2, , n}, or (2) B is a (r + 5+ l)-design, or (3) B is a {1 r+ 8, t+ 5+2}- 
design. If C is a self-orthogonal binary code with all weights divisible by 4 then the result extends to 
codewords of any given weight The special case of codewords of minimi weight in extremal self-dual 
codes also follows from a theorem of Venkov and Koch; however our proof avoids the use of modular 
forms. 


Reduced Lists of Patterns for Maximum Likelihood Soft Decoding 

.Iakov Snyders Department of Electronic Communications. Control and Computer Systems, Tel-Aviv 
University, Ramat-Aviv 69978, Israel 

Maximum likelihood soft decision decoding of an ( n,k,d ) binary linear block code is pcrformable 
by complementing the hard-detected version of the received word in at most m = n-k positions. The set 
of positions, or pattern, is selected according to least sum of reliabilities of the associated bits. We 
introduce a method whereby out of all the patterns with cardinality m, m -1 and m -2 no more than, 
rcspecuveiy, 
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explicitly described patterns have to be scored, provided that d > 3. Here 5 is the Kroncckcr delta. This 
approach enables decoding of the (15,11,3) code by at most 51 real addiuons, compared to the previous 
best 83 additions required in the worst case by a different search scheme Decoding of the (31,26,3) and 
(32,26,4) codes is accomplished by less than 200 addiuons in the worst case, versus more than 1000 
additions performed by any previously published decoder Apphcatren of reduced lists of patterns to 
cosct-dccoding of medium rate codes is also addressed 


Bounds on Codes via Kolmogorov Complexity 

John T Coffey and Rodney M. Goodman California Institute of Technology, Pasadena, CA 9! 125 

In this paper, we examine the use of the theory of Kolmogorov complexity in analysis of the error- 
correction capabtliucs of vanous classes of codes (c.g , systematic linear codes, shortened cyclic codes, 
generalized Recd-Solomon codes) under vanous error-correction strategics (random-error correction, 
burst-error correction, and vanous mixed strategies). Many original results are given, in addition, the 
techniques used are new and intuitively appealing Some comments are added about the problem of 
giving explicit constructions for codes that meet these bounds, and we also give one additional useful 
result denvable from the pnnciplcs of Kolmogorov complexity 
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Bounds on the Dimension of Certain Codes and Subcodes 

Alexander Vardy, Jakov Snyders, and Yair Be'ery Department of Electronic Communications Control 
and Computer Systems, Tel-Aviv University, Ramat Aviv 69978, Tel-Aviv, Israel 

Let C be an (n,k) linear block code over GF (q), generated by a matrix G. A nonnegative integer A 
is said to be the contraction index of C if a maximal set of pairwise linearly independent columns of G 
has k +A elements In recent works of Conway & Sloane, Be’ery & Snyders and Fomey it was shown 
that the complexity of soft decision decoding of block and lattice codes can be considerably reduced by 
means of partitioning the code into cosets with respect to a subcode of large dimension and small 
contraction index. In this paper we derive several upper and lower bounds on the dimension of a subcode 
with a prescribed contraction index. We also present an upper bound on the dimension of any (n,k) code 
over GF(q), with minimum Hamming distance d and contraction index A. For certain values of n and d, 
the latter bound is shown to be tight for all q and A. This substantially generalizes the results obtained 
previously for A = 1 in the context of majonty logic decoding. 


Achieving the Cutoff Rate on Communications Channels 

J. T Aslanis and J. M Cioffi Information Systems Laboratory, Stanford University, California 

In this paper we claim that combining the best current coding techniques with appropriate 
equalization can achieve the cutoff rate on channels with memory at any SNR We shall examine 
charactensncs of codes, equalizers, and signal sets that both the cutoff rate and capacity formulas indicate 
are necessary for this performance The capacity and cutoff rate formulas indicate the optimal bandwidth 
and bit spectral efficiency (bits/symbol/Hz) for a given input signal power constraint We show that the 
infinite blocklcngth limiting performance of vector coding, an approximation to a multiple earner 
technique combined with trellis coding, nearly achieves the cutoff rate. We then bound the difference 
between capacity with and without power spectral shaping and note that this difference, which equals the 
vector coding power penalty, becomes important only at very low SNR In this range of SNR, below one 
bit/symbol, trellis codes cannot provide adequate coding gam, but convolutional codes (or block codes) 
can easily be combined with the vector coding "cqualizapon" techniques to achieve cutoff rate 
performance, with moderate complexity We show that, for a fixed SNR, the capacity and cutoff rate Ro 
differ by less than 1-1/2W2 bits per symbol on any channel with or without ISL. We finally note that, 
while the full 1.53 dB of shaping gain affects the cutoff rate at high SNR, the total gain cannot be 
obtained if one fixes the available bits/d'rncnsion in a multi-dimensional signal set 


42 


SESSION MP7 


ERROR-CONTROL AND OTHER CODING 


The Rate/Performance Tradeoffs of Focused Error Control Codes 

Tom Fuja and Fady Alajaji Department of Electrical Engineering, Systems Research Center, University 
of Maryland, College Park, MD 20742 

Let B be a set of non-zero elements of F q (q > 2); we say a code is (ri,r 2 }-focused on B if it can 
correct up to t{ + tz errors provided at most ri of those errors lie outside B. The strategy is to offer 
different levels of protection against "common” errors - those in B - and "uncommon” errors. (The 
motivating example: correction of a single-bit-per-byte errors with codes over F? ) 

This lak will compare the performance and rates of (fi,r 2 )-focused codes with those of traditional 
t[ + r 2 -error correcting codes. We show that, at high SNR, if a channel is sufficiently "skewed" - that is, if 
the noise character is Z and P[Zi B |Z 5*0} </„„ - then the performance of a (r|f 2 )-focused code is 
essentially idennea! to that of a t\ + t 2 - error correcting code; this claim is derived analyucally and 
verified by simulation results Since (ti,r 2 )-focuscd codes can be constructed with higher rates than can 
r, +r 2 -crror correcting codes, they offer for these "skewed” channels new advantages m terms of rate 
and/or performance We include in the talk an analysis of the tradeoffs offered by focused codes for M- 
ary PSK and A/-ary ASK modulation schemes 


Design and Implementation of Binary Combined Error Control and Line Coding: a BCH-based 
Example 

1 J. C'ReiUy, S. Williams, and A. Popplcwcll School of Electronic Engineering Science, University of 
Wales, Bangor, Dean Street, Bangor, Gwynedd LL5 7 WT, United Kingdom 

There has been considerable interest expressed recently in defining transmission codes, which 
combine error control and line coding features both for telecommunications transmission and magnetic 
recording. This paper introduces the design of such a novel combined error control and line coding 
scheme, and desenbes its successful practical rcalizauon using a BCH example The design proceeds by 
constructing linecodcs from known error control codes and in this way we arc able to form a linccodc 
which is constnictcd entirely of valid error control codewords which thus inherits its error control power 
The design is applicable to a wide range of error control codes, but attention is focused on its applicauon 
to standard BCH codes when considering practical implementation The implemented system is 
functionally described and the power spectra consequently produced by ns encoded output arc presented 
and compared with those denved by theoretical analysis It is seen that these exhibit the charactcnsuc 
DC-null of a convenuonal linccodc 


The Design of Error Control Codes for Rayleigh Fading Channels with Memory 

Jean-Claude Bclfiorc Ecole Nationale Superieure des Telecommunications, 46, rue Barrault, 75634 

Paris Cedex 13, France 

The aim of this paper is to give a method to design codes adapted to the Rayleigh fading channel 
with memory. We first derive a Chcmoff bound of the bit error probability after decoding This bound 
shows that coding yields an equivalent order of diversity which increases with the average signal-to-noise 
ratio until a limit value which is the minimum (or free) distance of the code. We show that, in order to 
minimize the BER, the transmitted symbols must have unequal energy. The optimal energy distribution 
has been computed for some short block codes It provides important gains over the uniform one 
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Error-Control Coding for the Binary JV-user Modulo-^ Channel 

Vivek Telang and Mark Herro Department of Electrical and Computer Engineering, University of Notre 
Dame, Notre Dame, IN 46556 

Efficient use of transmission facilities often requires file sharing of resources by a number of users. 
Commonly used methods of shanng resources are Time Division Multiplexing (TDM) and Frequency 
Division Multiplexing (FDM). In this paper, we describe a different approach to designing an A'-user 
Multiple-Access System for the Binary Modulo-^ Channel. We design a coding scheme for N users, such 
that when all N users are sending data, the channel is used at its maximum capacity However, when 
fewer than N users are active, the unused channel capacity is used to achieve error control The extent of 
error control is inversely proportional to T, the number of active users. The multiple-access codes 
described in this paper are based on well-known error-correcting codes, viz., BCH codes, Reed-solomon 
codes and convolutional codes. Other codes that are being currently studied are cyclic codes and some 
Unequal Error Protection (UEP) codes. Possible applications of these codes are in a multiple-access 
system used by data sources as well as voice sources The codes exploit the 50% activity of a typical 
bursty voice source to afford error-piotecuon for the data sources without requmng additional channel 
bandwidth. 


On the Decoder Error Probability of Linear Codes 

Kar-Ming Cheung Communication Systems Research Section, Jet Propulsion Laboratory, 4800 Oak 
Grove Dr, Pasadena, CA 91109 

In a recent paper by the author, an explicit formula which enumerates the complete weight 
distnbutton of an (n.Jk) linear block code using a partially known weight distnbution is derived Also an 
approximate formula for the weight distnbution of most linear block codes is given, and is shown to be 
approximately binomial. 

In this paper, by generalizing the coding and combmatonc techniques mentioned in the above 
paper, an approximate formula for the weight distnbution of dccodabic words of most linear block codes 
is evaluated This formula is then used to give an approximate expression for the decoder error 
probability Pe(u) of linear codes, given that an error pattern of weight u has occurred It is shown that 
P E (u) approaches the constant Q as u gets large, where Q is the probability that a completely random 
error pattern will cause decoder error. 


Construction of Linear Codes of Minimum Distance Five 

C. L Chen IBM Corporation, P 0 Box 950, Poughkeepsie, NY 12602 

In this paper, wc present a method of constructing binary linear codes that have a minimum 
Hamming distance of five Some of the new codes obtained arc more efficient in information rates than 
other known codes 


The Permutation Channel 

Jonas Wallbcrg and Ingemar Ingemarsson Department of Electrical Engineering, Linkoping University, 
S-581 83 Linkoping, Sweden 

Permutation Modulation, PM, is a seldom used modulation scheme it has however many favorable 
features like: each codeword requires the same energy, an easily lmplemcntablc maximum likelihood 
receiver, the probability of an incorrect detection is the same for each sent signal 

We will here describe a special abstract channel wiuch wc call the Permutation Channel. The 
Pcrmutauon Channel contains the Additive White Gaussian Noise Channel The vector model of the 
AWGN Channel is used. The set of transmitted vectors is obtained by permuting the components of a 
given initial vector in all possible ways The PM detector maps the received vector onto the set of 
transmitted vectors. 
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For this channel we have computed upper bounds on the probability for different kinds of 
permutation errors. These error probabilities are important when constructing codes for the Permutation 
Channel. The performance of Permutation Modulation and Peimutation Codes is compared to the 
performance of ordinary block codes with amplitude and phase modulation. 


A Class of Error Correcting Codes for the Permutation Channel 

Jonas Wallberg and Ingemar Ingemarsson Department of Electrical Engineering, Dnkoping University, 
S-SSI S3 Unkoping, Sweden 

The permutation channel is a discrete, mcmoryless channel which permutes the order of the 
transmitted components of a vector (see contnbution "The Permutation Channel") A Permutation Code 
is a subset C of the set of vectors fi (in the Euclidean space) obtained by Pcimutauon Modulauon, 
variant 1. Thus a codeword is desenbed by a permutation, p, of the components of the iruual vector sq. 
The enor correcting codes will be 
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TECHNICAL SESSIONS 


Tuesday, 9 a.m. -12 m. 
SESSION TA1 

COMMUNICATION THEORY I 


A Model for the Statistical Analysis of Sigma-Delta Modulation 

Pmg Walt Wong and Robert M. Gray Dept, of Electrical and Computer Engineering, Clarkson 
University, Potsdam, NY, 13676, and Information Systems Laboratory, Dept of Electrical Engineering, 
Stanford University, Stanford, CA, 94305 

A stochastic model is suggested that describes the behavior of a single-loop sigma-delta modulator, 
which is an oversampled feedback quantization scheme. When the input to this nonlinear system is ar 
11 d, Gaussian process, the model can be represented by a Wiener process embedded in a renewal process 
A condmon is given so that the difference between the output predicted by the model and that of the true 
system can be made arbitranly small. Using this model, v/e can show the convergence and mixing 
properties of the output of the single-loop sigma-delta modulator The power spectral density of the 
output sequence can also be derived (Research supported by NSF Grant MIP87-06539 and a Stanford 
University Center for Integrated Systems seed grant) 


Error Probability for Digital Transmission over Nonlinear Channels with Application to TCM 
Yow-Jong Liu, Ikuo Oka, and Ezio Biglicn Electrical Engineering Department, University of California, 
Los Angeles. CA 90024-1594 

We consider the computation of error probability for digital transmission over nonlinear channels 
with a finite memory It is well known that trcllis-codc modulauon (TCM) encoders can be modeled as 
nonlinear systems with finite memory To prove that the converse is also true, we use orthogonal Volterra 
senes to denvc a "canonical" rcpresentauon for discrete nonlinear systems, based on a linear 
convoluuonal code and a mcmorylcss mapper This representation shows that finite-memory discrete 
nonlinear systems can be analyzed in much the same way as TCM schemes In particular, TCM over 
nonlinear channels can be analyzed. 

The "pairwise suite" technique is in pnnciplc applicable to the computation of error probabilities for 
nonlinear channels It involves matnees, where N is the number of states in the trellis 

representation of the overall channel If TCM over a nonlinear channel is considered, N is the product of 
the number of code states and the number of channel suites, and this may be very large 

We denve an upper bound to the error probability that avoids the considcrauon of N 1 x matnees 
It is based on the computation of the transfer function of a graph with N +1 nodes, whose branch labels 
are matrices rather than scalars. As a result, consideration of situations involving a relatively large 
number of suites is made possible, A lower bound to the error probability is also denved. Examples of 
application to the analysis of a number of situations involving nonlinear channels arc also provided 


Probability Distribution of DPSK in Tone Interference and Applications to SFH/DPSK 

Q, Wang, T, A, Gulliver, and V, K. Bhargava Department of Electrical and Computer Engineering, 

University of Victoria, P 0 Box 1700, Victoria. B.C., Canada V8W Q2Y2 

Siow frequency hopped differential PSK (SFH/DPSK) has been the subject of much recent 
attention. The most salient feature of SFH/DPSK is that it can sustain a much higher data rate than a fast 
frequency hopped system with the same hop rate. We present a study of the probability distribution of the 
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received DPSK signal under tone jamming to facilitate the analysis of a SFH/DPSK system. 

The results given are more general than those previously published in several aspects. First, the 
differential phase of the transmitted DPSK signal can assume any value. Second, probability distnbutions 
are derived instead of a set of probabilities calculated over certain symmetrical regions. This allows 
maximum flexibility in performance analysis as far as decision regions m demodulating DPSK are 
concerned. Third, the joint probability distribution of both the magnitude and differential phase of the 
jammed DPSK signal is given. This can be used in the analysis when both tone jamming and Gaussian 
noise are considered. To illustrate the application of these results, we analyze the error probability 
performance of a general uncoded SFH/DPSK signal under worst case tone jamming and Gaussian noise. 


Estimation Variance Bounds of Importance Sampling Simulations in Digital Communication 
Systems 

D. Lu and K. Yao Electrical Engineering Department, University of California, Los Angeles, Los 
Angeles, CA 90024-1594 

In practical applications of importance sampling (IS) simulauon, we encounter two basic problems, 
that of determining the estimation vanance and evaluating the proper IS parameters needed in the 
simulations. We derive a simple upper bound on the esumation vanance which is applicable to all known 
importance sampling techniques. Furthermore, a lower bound on the improvement ratio of vanous 
importance sampling technique relative to the direct Monte Carlo simulation is also given. These bounds 
are shown to be useful and computationally simple to obtain Based on these bounds, we can readily find 
the needed sub-optimum IS parameters Specific numencal results indicate that these bounding 
techniques are applicable to many problems involving Gaussian and non-Gaussian pdf's, linear as well as 
non-linear communication systems with ISI in which exact bit error rates and IS estimation vanance 
cannot bo obtained simply by previously know . approaches. 


Bit Error Simulation via Conditional Importance Sampling 

Tao Chen and Charles L. Weber Communication Sciences Institute, Department of Electrical 
Engineering-Systems, University of Southern California 

A new vanation of importance sampling, designated as conditional importance sampling, or CIS, is 
proposed for the simulation of bit error rate in nonlinear digital communication systems. Monte Carlo 
simulations are simple and tractable m simulating the bit error rate, but is prohibitively slow when errors 
are rare. Importance sampling is a variauon of Monte Carlo which increases the simulation speed by 
altcnng the input density functions and weighting the output to have an unbiased estimator Earlier 
research concentrated on a fixed amouni of bias in the input noise probability density function, or pdf, for 
every simulation trial. We extend the biasing to the phase error pdf, and, in order to utilize more of the 
knowledge we have about the system, we suggest conditional importance sampling CIS biases the pdf of 
some input vanables according to the values of the other ones. The estimator variances of CIS arc 
evaluated for some simple systems, and the resulting improvements over standaa Monte Carlo arc 
significant. 


A Contribution to the Proof of the Simplex Conjecture 

Dejan E. Lazid Universitat Karlsruhe, Institut fur Algorithmen und Kognitive Systeme, D-7500, 
Karlsruhe /, Fed Rep. of Germany 

In order to prove the simplex conjecture (the set of M=N +1 unit vectors in //-dimensional 
Euclidean space that form a regular simplex is the opUmum error protecting code for memoryless-channel 
with discrete time and addiuve Gaussian noise), two lemmas are established in this paper. The first one 
purports that the probability of error for a Voronoi cell (a radial projection of the maximum likelihood 
region to the surface of a unit //-dimensional sphere Lip) that forms a regular spherical simplex on Lin, 
that has N sides, and whose center ot inscribed spherical cap contains a corresponding code word (this 
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cell will be called optimal Voronoi cell), is always less than the probability of error for another Voronoi 
cell of the same area as the optimal one, regardless of the position of its corresponding code word in its 
interior. The second lemma asserts that the average probability of error for optimal Voronoi cells is the 
least when all their areas are equal, given that the total area is constant. Lemma 1 is proved for At =3 and 
N=4, and lemma 2 for every N>1, establishing the validity of the simplex conjecture for N =4 (till now 
the conjecture was proved for V <4) 


Joint Synchronization and Detection from Multiple Samples per Symbol 

Costas N. Georghiades and Marc Moeneclaey Electrical Engineering Department, Texas A&M 
University, College Station, TX 77843, and Communications Engineering Lab, University of Ghent, 
Belgium 

We invesugate the problem of sequence estimation in the presence of a uming uncertainty from 
samples of the output of a matched filter taken at an integer muluple of the symbol rate. The effects of 
sampling rate and signaling pulse shape on both delay and sequence estimauon performance are studied 
by means of lower bounds on mean-square estimation error, ar.d error-probability respectively. Further, 
algonthms for delay and sequence estimation, based on joint maximum-likelihood processing of the data 
samples, and an iterative algorithm motivated by the expectation-maximization (EM) algorithm are 
introduced. The general results are applied to the case of binary antipodal signaling and rectangular pulse 
shapes 


Performance Analysis of the Sequential Algorithm for Intersymbol Interference Channels 
F Xiong and E Shwedyk Department of Electrical Engineering, University of Manitoba, Winnipeg, 
Manitoba, Canada R3T 2N2 

A receiver consisting of a whitened matched filtu (WMF) followed by a sequential algorithm (SA) 
has been developed for intersymbol interference (ISI) channels The channcl-WMF system can be 
modeled as a finite state machine (FSM) The present work gives performance analysis The symbol error 

probability is given by where constant K depends on the channel; a is the spectral density of 

2cr 

the Gaussian noise, d m „ is the minimum distance between received data sequences. To determine the 
computational complexity, the FSM can be interpreted as a special convolutional encoder followed by a 
binary symbol to 2-ary symbol mapping (2 = 2 L , L is the length of the channel impulse response) It 
follows that the computational distnbution is Pareto and there exists a computational cutoff rate R comp 
For die uncoded data considered the rate is fixed and the entenon translates into a signal to noise 
(SNR) entenon, i.e., for bounded computation per node the SNR should exceed SNR cimp . An upper 
bound SNR' ca ^ l on SNR comp is found analytically by assuming a uniform input distnbution Computer 
simulation results venfy the above analysis and show that for a one-pole channel the SA is 1-3 dB better 
than the Af-algonthm 


Minimum Error Probability for Asynchronous Multiple Access Uncorrelated Fading Intersymbol 
Interference Channels 

Daoben Li Dept of Information Theory, Beijing University of Posts and Telecom.,lunications, Beijing 
100088, China 

Consider an L-user muluple access digital communication systcm.in which all the subchannels 
corrupted by addiUve white Gaussian noise are uncorrelated fading intersymbol interference channels 
with the same staustical properties, and the users transmit independent data streams by modulaung 
anlipodally a set of assigned signal waveforms without maintaining relative bit-synchronism among 
them. The minimum error probability achieved by the maximum-likelihood sequence detectors is studied 
first The corresponding optimum signal set and their physical realizability are next studied. It is shown 
that if the inphase and quadrature components of the signals are complemenlanly like a group of 
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bandlimited white noises, there is no performance degradation due to the presence of other users, and the 
intersymbol interference is a beneficial factor. The wider the system bandwidth or the larger the time 
dispersion of the channels, the better the system performance. In the limiting case the multiple-access 
data-transmission system with intcrsymbol-interference channels would approach that of a Gaussian 
single-use channel. 
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SESSION TA2 


MODULATION 


A Coded Modulation Scheme with Interblock Memory 

Mao-chao Lin Department of Electrical Engineering, National Taiwan University, Taipei 10764, 
Taiwan, ROC 

We present a block coded modulation scheme for which the coded modulation of each block 
depends on the preceding block. The design is a combination and modification of Sayegh’s coded 
modulation scheme and a Sloane-Reddy-Chen’s binary code construction. Using signal set geometry of 
8-PSK and 8-AMPM, we design coded modulation systems with code rates of 29/45 and 15/24 
respectively. The asymptotically coding gains are 6.02 dB and 5.05 dB compared to the uncoded 4-PSK, 
for which the decoding complexity are respectively similar to those for the 32-state and 16-state trellis 
coded modulations designed by Ungerboeck. If we consider the effect of the number of nearest neighbors, 
the coding gains are 5 34 dB and 4 52 dB rcspecuvely 


Combined Coding and Modulation Using Block Codes 

Klaus Huber Institut fur f'-tzwerk- und Signaltheone, Techmsche Hochschule Darmstadt, Merckstr 25, 
6100 Darmstadt, Wcst-Germany 

In this contnbuuon we present combined coding and modulation schemes which are simple, 
powerful, and easy to synchronize. At the same data rate and power consumption they improve 
considerably over uncoded signaling schemes The csscnual new idea is the contraction of a no bit- 
sequence to a (ng-/) signal point-sequence, giving a redundancy of / bits at our disposal, which can be 
used for example for error control coding An important advantage is, that the schemes are easy to 
implement Also the problems of synchronization are much better to overcome. 


Modulo Signu-Delta Modulation for a Random Process Input 

Wu Chou and Robert M Gray Information Systems Laboratory, Department of Electrical Engineering, 
Stanford, CA 94305 

Modulo sigma-delta modulation (MSDM) has ns important application in communication and 
ovetsamplcd analog-lo-digital conversion. MSDM is formed by having a modulo-limiter as the front end 
to compress the input into the non-overload region of the sigma-delta modulator An exact analysis cf the 
response of this nonlinear system with regard to an input with independent increments has been obtained. 

It is shown that the normalized binary quantization noise process e, of MSDM, when the input is a 
process with independent increments, is a quasi-stationary process It is uniformly distributed in 
[-1/2, 1/2] It is asymptotically white and uncorrclatcd with the input This is a significant difference 
from the case of dithenng where the binary quantization noise is never white 

The method wc use here provides an effective way to decide the asymptouc dislnbuUon of the 
quantization noise and its spectrum It can also be applied to other modulation schemes such as modulo- 
PCM, modulo-DPCM and modulo multi-stage sigma-delta modulauon 


Embedded Modulation and Coding for HF Channels 

B Honaiy and M. Darnell Department of Engineering, University of Warwick, Coventry, CV4 7AL, UK 
and Department of Electronic Eng , University of Hull, Hull, HU6 7RX, UK 

In the proposed paper a number of new concepts and techniques intended for application to radio 
systems operaung over time variable channels will be discussed These arc currently being investigated 
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via several co-ordinated research programs by the authors, with the authors, with the aim of integrating 
them progressively into the architectures of adaptive HF radio systems. However, it should be stressed 
that the techniques are not specific to HF radio: in principle, they could also be applied to other forms of 
radio system with non-Gaussian noise and interference backgrounds. 

The techniques, which are studied and simulated m software form are implemented using Digital 
Signal Processor) (i) a new method for implementation of embedded codes, (it) embedded modulation, 
and (in) real-time channel evaluation obtained by varying constellation angles 


Design of (0,1) Sequence Sets for Pulsed Coded Systems 

F. Khansefid, H. Taylor, and R. Gagliardi Communication Sciences Institute, University of Southern 
California, Los Angeles, CA 90089 

A (0,1) sequence is a sequence of zero (off) and one (on) binary symbols of a given length. A set of 
sequences is a group of such (0,1) distinct sequences The sequences can be considered as pulse 
waveforms, in which the one symbol corresponds to the location of an electronic or optical pulse while a 
zero prepresents a zero, or pulse absence. The design of (0,1) sequence sets therefore correspond to the 
design of families of pulsed waveforms that can be used as a codesct, and therefore has application to on- 
off pulse signaling in communication and signal processing systems. 

This paper presents the results of a study relating the above sequence parameters to the ability to 
produce good autocorrelation and cross-correlation properties. Sequence sets having the optimal 
correlation properties are derived and tabulated for various parameters combinations. Best possible 4 - 
pulse sequences with pairwise correlation <, 1 have been found where the numbers of sequences in the set 
is any number up to 14 For 5 - pulse and 6 - pulse sequences, results close to the Klove bound have been 
derived A general construction is given for such sets, and the results of a computer search for optimal 
code sets is reported. Design bounds on the set parameters are also presented 


VLSI Viterbi Decoder for a BCM Code 

Roksana Borch, David Coggins, Branka Vucctic, and Shu Lin Laboratory for Comm Set and 
Engineering, Sydney University Electrical Engineering, NSW 2006 Australia, and University of Hawaii, 
Manoa, HI, 96822 

This paper analyzes the VLSI implementation of a high-speed Viterbi decoder for a 4-statc Block 
Coded Modulation (BCM) code with 8-PSK symbols. It demonstrates that practical decoders of low 
complexity for TDMA applications can be built on one chip for 1004/ Bps data rates. The device is being 
implemented in 1 5 pm CMOS technology The problems of synchronization, quantization and metric 
calculation are investigated A comparison in terms of speed and decoder complexity is made on the 
register exchange and traccback methods, and it is shown that, for this particular code, the former is more 
suitable 


On the Capacity of the Gaussian Channel with a Finite Number of Input Levels 
L, H OzarowandA D Wyner AT&T Bell Laboratories, Murray Hill, NJ 07974 

We give an analytic confirmation of an observation made by Ungerbocck (in "Channel Coding with 
Multilcvcl/Phase Signals," IEEE Trans, on Information Theory, Vo! IT-28, Jan. 1982, pp. 55-67) that 
approximately optimal performance on a Gaussian channel with capacity C can be obtained with about 2 C 
levels In particular our results imply that by using about 2 C ~' levels, we can achieve a rate of nearly 
C -1 bit, and that by using about 2 f f ' levels we can achieve a rate of about C -0 4 bits 


Enumerative Coding for Constrained Noiseless Channels and Modulation Coding 
Boris Fitingof Optical Sciences Center, University of Arizona, Tucson, AZ 85721 

Practically any group of constraints defining req' .rements for modulation codes can be described by 
a state transition table (STT) with finite number s of states (e.g, for any nm-length-hmited (d,k) code 
r = k +1). Synchronization words can be easily introduced as additional constraints by modifying STT. 
New coding-decoding algonthms for constrained noiseless channels are proposed. These algorithms have 
been implemented in C under UNIX For any given STT and any given length L of codewords the 
algorithms satisfy modulation constraints defined by this STT and provide optimal rate under these 
modulation constraints and for given L. Hence the rate converges to the Shannon’s capacity for the 
noiseless channel when L -* °°. For the case of the run-length-limited codes computer expenments show 
rapid convergence of the rate to the capacity with increasing input block length. The proposed algonthms, 
for both coding and decoding, use as an additional input an array of sxL integers completely 
determined by the STT and L, Uj practically do not use other memory besides this array, (3) use the 
number of operations of order s per bit of encoded data, (4) achieve aforementioned results by 
enumeration of all output words satisfying the modulation constraints, using the enumeration algonthm 
for decoding and an algonthm inverse to the enumeration algonthm for coding. 
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SESSION TA3 


OPTICAL COMMUNICATIONS 

Trellis Codes for the Optical Direct-Detection Channel 

Gregory J. Potne Codex Corporation, 20 Cabot Blvd, Mansfield, MA 0204S 

It is shown that the reduced complexity minimum distance search method of Zehavi and Wolf can 
be extended from codes designed using the squared Euclidean distance measure to any measure which is 
additive over the code branches. Recently Georghiades has derived a distance metric for codes for the 
direct detection optical channel. The method is applied to find codes for four and eight point overlapping 
pulse position modulation (OPPM), with large minimum distance and a small number of codewords at 
that distance. 


Application of Quantum Minimax Rule to General Ternary Quantum State Signals 
Masahiko Sekiguchi, Osamu Hirota, and Mar.ao Nakagawa Faculty of Science and Technology, Keio 
University, 3-14-1, Hiyoshi, Hohoku-ku, Yokohama 223, Japan, and Faculty of Technology, Tamagawa 
University, 6-1-1, Machida, Tokyo 194, Japan 

A receiver which minimizes an effect of quantum noise is called quantum opumum receiver in 
opucal communications. So far, the quantum Bayes rule and Neyman-Pearson rule were formulated by 
Hetstrom. These rales are attractive, but very difficult to apply to arbitrary quantum state signals, except 
for binary signals and a few specific ones. On the other hand, the quantum minimax rule was foimulated 
by Hirota, which provides a simplification of the calculation for the quantum detection problems. 
However, it seems to us that his method i. still complex in order to solve the problems for arbitrary 
quantum signals. 

In this presentation, a new calculation method of the quantum imiumax rale for the pure state 
signals is proposed, employing a gcomctncal interpretation The calculation process of N 1 equations 
obtained from the necessary and sufficient condit'ons of the minimax rale is greatly simplified by our 
method. As an example, the general solution for ternary signals is given Then it is numcncally shown 
that the optimum receiver based on the minimax rule is always supenor to the quasi-classical receiver in 
case of the general ternary signals 

Error Probabilities in Optical Receivers with Avalanche Diodes 

Clua Lu Ho and Can W Helstrom Dept, of Electrical & Compeer Engrg., R-007, University of 
California, San Diego, La Jolla, Calif 92093 

Error probabilities in optical receivers using avalanche diodes are calculated by numencal 
integration of Laplace reversion integrals in the complex plane along paths passing through a saddlcpoint 
(The communication system is transmitting information coded into equally probable 0's and l’s, 
represented respectively by a blank and by the transmission of a light pulse along a fiber) The integrand 
involves the moment-generating function of the amplified output of the diode, and this is calculated from 
Pcrsonick’s model of the avalanche multiplication process We take into account the shape of the 
incomuig light intensity pulses, including iniersymbol interference, and consider both exponential and 
Gaussian current pulses at the amplifier output. 

Error Probability of Optical Feedback Receivers 

Goran Erearsson Telecommunication Theory, Lund University, Box 118, S-22I 00 Lund, Sweden 

An analysis of noncoherent optical feedback receivers is presented The signal statistics at the 
decision point are derived and the error prooability is determined. 
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The negative feedback generates a shot noise process which is the difference between two Poisson 
point processes. The probability distnbution of such a process is studied and compared with Gaussian and 
Poisson distributions. 

The error probability is calculated for an ideal receiver model consisting of a feedback amplifier 
with freq - :ncy-independent gain followed by an integrate-and-dump filter A comparison is made 
between the exact error probability and the results obtained from a Gaussian approximation It is shown 
that the straight-forward Gaussian approximation works well and gives results close to the exact values. 

A more realistic receiver model with arbitrary filter is studied. A saddlepoint approximation is 
derived which can be used to calculate the error probability with high accuracy 


Electrical Signal Processing Techniques in Long-Haul, Fiber-Optic Systems 

Jack H. Winters and Richard D. Gulin AT&T Bell Laboratories, Holmdel, New Jersey 07733 

The purpose of this paper is to demonstrate the potential for electrical signal processing to mitigate 
the effect of intersymbol interference in long-haul, fiber-optic systems. Intersymbol interference in long- 
haul fiber-optic systems can severely degrade performance and consequently limit both the maximum 
distance and data rate. The sources of intersymbol interference include nonlineanty in the transmit laser,, 
chromatic dispersion tn systems operated at wavelengths other than the dispersion minimum of the fiber,, 
polarization dispersion, and bandwidth limitations in the receiver. We discuss several techniques for 
reducing intersymbol interference in single-mode fiber systems with single-frequency lasers and show 
which techniques are appropnatc at high data rates in direct and coherent detccuon systems In particular, 
we ana'.vze the performance of linear equalization (tapped delay lines), nonlinear canccllauon (vanable 
threshold detection), maximum likelihood detection, coding, and multilevel signaling. Our results, for a 
simulated binary 8 Gbps system, show that simple techniques can be used to substantially reduce 
intersymbol interference, increasing system margin by several dB. A novel, but simple, nonlinear 
cancellation technique can more than double the dispersion-limited distance and/or data rate. 


Nonparametric Inference for a Doubly Stochastic Poisson Process 

Klaus Utikal Department of Statistics, University of Kentucky, Lexington, KY 40506-0027 

Consider a doubly stochastic Poisson process whose intensity X, is given by A, = a(Z,), where a is 

a unknown nonrandom function of an information process Z, Only one continuous time observation of 

: 

counting and information process is available The function A (z) = j a(x) dx is estimated from the class 

o 

of Lipschitz continuous functions a The normalized esumator is shown to converge weakly to a 
Gaussian process as lime approaches infinity Confidence bands for A arc given Tests for independence 
from the process Z, arc proposed 


Analysis of Coherent Random-Carrier CDMA and Hybrid WDMA/CDMA Multiplexing for High- 
Capacity Optical Networks 

B. Ghaffari and E Geramotis Department of Electrical Engineering & Systems Research Center, 
University of Maryland, College Park, MD 20742 

In this paper we provide an exact analysis of the performance of a random-carrier (RC) code- 
division multiplexing (CDMA) scheme recently introduced for use in high-capacity optical networks. 
According to this scheme coherent optical techniques are employed to exploit the huge bandwidth of 
single-mode optical fibers and are coupled with spread-spectrum direct-sequence modulation in order to 
mitigate the interference from other signals due to the frequency overlap caused by the instability of the 
carrier frequency of the laser, 

The average bit error probability of this multiplexing scheme is evaluated with a'bitrary accuracy 
by integrating the charactenstic functions of the components at the output of the matched optical filter 
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due to the desired signal and the other-user interference. Both phase noise and thermal noise (AWGN) are 
taken into account in die computation. The analysis is valid for any spreading gain and any number of 
interfering users and does not make use of limiting theorems and Gaussian approximations. The 
performance evaluation of RC CDMA establishes the potential advantage in employing hybnds of 
frequency-division and code-division multiplexing to combine the best features of both multiplexing 
schemes. (This research was supported in part by the Office of Naval Research under contract N00014- 
89-J-1375 and in part by the Systems Research Center at the University of Maryland, College Patk, 
through the National Science Foundation's Engineering Research Centers Program: NSF CDR 
88003012) 


Soft Maximum Likelihood Detection for Balanced Binary Block Codes 

Michael Hall and Gareguin S. Maikanan Communications Laboratory, Helsinki University of 
Technology, Otakaari 5A, SF-02150 Espoo, Finland, and Radiophysik and Electronic Institute, Armenian 
Academy of Sciences, 378410, Ashtarak-2, Armenia, USSR 

In this paper, a simple and fast soft decision decoding algorithm is proposed The algonthm reduces 
the probabdity of error per code word in balanced nB-(n+l)B codes, which are often utilized in digital 
fiber-optic transmission systems generally employing symbol-by-symbol hard decision, without greatly 
increasing the complexity of the receiver 

Using a 1B-2B and 3B-4B code as an example, it is shown Jiat the algonthm is simple to 
implement in practice. The soft decision algonthm decoder of the 3B-4B code can be designed to be 
implemented using only relatively few compansons, as compared to other soft decision decoding 
algomhms. The operauon of the 3B-4B decision algonthm is presented in detail in the paper. 
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SOURCE CODING I 


A New Example of Optimal Source Coding for Infinite Alphabets 

Julia Abrahams Mathematical Sciences Division. Office of Naval Research, Arlington, Virginia, 22217- 
5000 

The Huffman code is found for a particular integer source. The probability distribution of the source 
alphabet is given by 

p, =(l-0X0/2) flo,, ‘ l , 1=1.2, ••• 

for 0.390 £ Q 0.618. The optimal code has a length vector given by 

/, = 2[log 2 i] + 1. i=l,2, • • • 

([x ] is the largest integer less than or equal to x.) 

The method appears to be amenable to other parametric families of distributions. For a parametric 
family which includes a highly structured dyadic distribution, the length vector can be determined from 
that dyadic distribution. Numerical evidence suggests the form of the code tree to be verified as optimal 
by means of Gallagcr’s sibling property. Ideally the sibling property is equivalent to a few inequalities 
that determine the applicable parameter range. This approach extends the class of integer alphabets for 
which optimal binary codes arc known. Previously only the geometric and Poisson alphabet cases were 
solved. 


Optimization of Overlapping Block Transform for Source Coding 

Miodrag Temerinac and Bemd Edler I ns taut fur Theoretische Nachrichtentechnik und 
Informationsvcrarbeitung. Universitdt Hannover (FRG) 

Transform coding with or without block overlapping (TO and subband coding (SBC) are two 
methods for bit rate reduction of correlated signals. In this paper a common theory for both of these 
methods is applied, which is called "L into N coding’ (LINC), since both methods map L input signal 
samples into N output coefficients. Several realizations of LINC with exact reconstruction properties are 
known. They are either restricted to certain values of N or L, or they need complex design and 
optimization strategies. For example Conjugate Quadrature Filters (CQF) and Generalized Quadrature 
Mirror Filters (CQMF) arc restricted to N - 2 and higher values of N are only possible by cascading these 
structures. The transform with Time Domain Aliasing Cancellation (TDAQ and the Lapped Orthogonal 
Transform (LOT), on the other hand are restricted to L = 2 N. 

It is shown in this paper, that the overlapping block transform (OBT) is a suboptimal solution of 
LINC with arbitrary values of L and N. Furthermore, a fast algorithm atvd a simple optimization strategy 
based on effective bandwidth minimization are presented. The proposed optimization method for OBT 
can be done independent of signal statistics and provides a high coding gain. The optimization results for 
2 £ N £ 32 and N £ L £ %N are analyzed and compared with those of a DCT on one hand and theoretical 
bounds on the other hand. The OBT provides a coding gain, which is higher than that of DCT. is 
independent of the correlation sign, and almost reaches the theoretical bounds. 


On Entropy Rate for Source Encoding on a Pyramid Structure 

R. P. Rao and W. A. Pearlman Electrical, Computer and Systems Engineering Department, Rensselaer 
Polytechnic Institute, Troy, NY 12180 

Entropy is defined in terms of the source spectrum and is used to analyze pyramid structures. Two 
different pyramid structures are considered - the difference pyramid and the orthogonal pyramid. 
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Conditions, in terms of spectral entropy, arc derived under which a pyramid structure is optimum. The 
Laplacian pyramid is also studied as a special case of the difference pyramid and theoretical justification 
for its optimality is given. The orthogonal pyramid has the property that spectral entropy of the pyramid 
is equal to that of the full-band signal and this property is used to prove that the two are equivalent in 'he 
rate-distortion sense. It is also shown that the combined hrst-order entropy of an optimally filtered 
pyramid is closer to the entropy rate of the source than the first-order entropy of the full-band signal. This 
means that loss-less compression of the source sequence can be obtained by merely representing it on a 
pyramid structure. Expenmental results using both synthetic sources and speech samples are presented to 
verify the above claims. 


A Combined Source and Error Correcting Code 

D Taipaie Department of Electrical and Computer Engineering, The University of Texas at Austin, 
Austin, TX 78712-1084 

When signaling over some channels (such as on-off keying on a fiber optic link), die error 
probability can be reduced by reducing the a-pnon probability of a pulse. In systems which use error 
correcting coding, this is difficult because most error correcting codes usually have as many ones as zeros 
(on the average). In this paper, we develop an algonthm which can be combined with any linear, cyclic 
code to obtain a combined source and error correcting code with any specified average symbol probability 
distribution The main idea in the algorithm is simple, each codeword is used as a distinct symbol m 
source code output alphabet The source encoding algonthm we use is a practical form of the Elias code 
developed by Jones. This algorithm requires that we order the codewords, and that for each codeword, we 
know the total number of ones in the codewords that occur earlier in the ordenng (the weight) For codes 
with a large number of codewords, it is not convenient to determine the weight using table lookup. To 
overcome this problem, we develop a way to find the weight (and so compute the correct codeword) one 
coordinate at a time 


Universal Source Coding with Order Information 

Kenneth Keeler Harvard University, Pierce Hall, Cambridge, MA 02138 

Consider the class n (HA, <.) of ud sources (charactcnzed as probability distributions p) of symbols 
from a countably infinite partially ordered alphabet (HA, <.) which obey the order in the sense that 
x £• y =>p(x)ip(y). (Decreasing distnbutions on the posiuvc integers (N, <) are the most basic 
examples) This paper addresses the problem of developing efficient source codes for A given only that 
the true source p is in Q(HA, £.). generalizing the work of Elias and Rissanen on representations of and 
universal codes for S1(N, <) A code is considered to be universal if its length funcUon f minimizes the 
worst-case inverse coding efficiency E p flH(p) over p e n(HA, <•) in the high-entropy limit. The 
existence, behavior and performance of universal codes for (HA, <.) are expressed in terms of the 
asymptotic behavior of the predecessor cardinality P(x) = | (y y <• x) | and the cardinality of its level 
sets; the charactcnzation is complete for the case of weak orders These results arc of particular 
importance in methods of inference by code length minimization, their application to which is discussed 
with illustrative examples. 


Combined Source-Cb"nnel Coding for Band-limited Waveform Channels 

V. Vaishampayan and N. Farvardm Electrical Engineering Department, Systems Research Center, and 
Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742 

In this paper, we consider the problem of optimum block-structured communication system design 
for a band-limited addiuve white Gaussian noise channel The transmitter consists of a block encoder, a 
signal selecUon unit and a modulator. The receiver consists of a demodulator and a source decoder. The 
objective is to design the source encoder, the modulauon signal set and the decoder so as to minimize the 
mean squared-error subject to constraints on the average transmitted power and the channel bandwidth. 




Necessary conditions for optimality are derived; however, it is difficult to solve these necessary 
condiuons in the most general case. For the special case where the modulation signal set belongs to the 
QAM family and for a maximum-likelihood (ML) demodulator, we have developed an algorithm that 
iteratively solves the necessary conditions for optimality for the encoder and the decoder subject to the 
above mentioned constraints. Numerical results are obtained for Gauss-Markov sources and different 
choices of system parameters. For comparison purposes, we have also determined the performance of a 
conventional vector quantization system combined with QAM modulation and ML demodulation. These 
results indicate that significant performance improvements over the conventional vector quantization 
system can be obtained. Furtheimore, comparisons are made against a system with a linear decoder in 
which the encoder, signal set and the linear decoder are optimized. Finally, vanous channel mismatch 
issues are studied and the rela'he robustness of the above systems are investigated 


Source and Channel Entropy Coding 

George H. Freeman Department of Electrical Engineering, University of Waterloo, Waterloo, Ontario, 
Canada N2L 3GI 

Entropy codes arc classified into two kinds. Source entropy codes such as common string parsers 
alter the source statistics to match a given channel Channel entropy codes such as the Huffman codes 
alter the channel statistics to match a given source The two kinds of codes are given dual descnpuons 
using parse/code trees. An heunstic and possibly optimal algonthm for the joint design of the trees yields 
codes for the binary 1 i.d. source having good noiseless compression with complexity much lower than 
either parsing or Huffman coding alone. The number of possible tree configurations is limited by the 
number of Huffman code trees Recursive expressions arc derived for the number of Huffman code trees 
with given height or given total number of nodes. 


Source Coder Structural Constraints and Information Patterns 

Jerry D. Gibson, Thomas R Fischer, and Wcn-Whci Chang Department of Electrical Engineering, Texas 
A&M University, College Station. TX 77843, Department of Electrical & Computer Engineering, 
Washington State University, Pullman, WA 99164, and Department of Communication Engineering, 
National Chiao-Tung University, Taiwan, R.O C 

Papers by Fine and Gibson and Fischer provide an approach for the design of data compression 
systems subject to a constraint on the transmitted alphabet. One step in this procedure employs what has 
been called the minimum search property Gabor, et al claim that this minimum search property may not 
be satisfied by optimal encoder/dccodcr pairs and present a purported counterexample with certain 
structural constraints. We show that the minimum search property is satisfied by cncodcr/decoder pans 
with classical information patterns, as required in the theory presented by Fine and by Gibson and 
Fischer, and that the counterexample is invalid since it has a non-classical information pattern imposed by 
the structural constraints. We then solve the example with a classical informauon pattern and the 
minimum search property. 


59 


SESSION TAS 


SHANNON THEORY H 


Diversity Coding for Transparent Self-Healing and Fault-Tolerant Communication Networks 
Ender Ayanoglu, R. D. Gitiin, Chih-Lin I, and J. E. Mazo Communication Systems Research Laboratory, 
AT&T Bell Laboratories, Crawfords Corner Rd, Holmdel, NJ 07733-1988, and Mathematical Sciences 
Research Center, AT&T Bell Laboratories, 600 Mountain Avenue, Murray Hill, NJ 07974-2070 

In this paper, a channel coding approach called diversity coding is introduced for self-healing and 
fault-tolerance in digital communication networks for nearly instantaneous recovery from n <M link 
failures. To achieve this goal, the problem of link failures is treated as an erasure channel problem Two 
methods are presented: (i) the parity generator matnx of the code is Founer, i.e, the panty lines are a 
discrete Founer transform of the data in GF( 2”), (ii) the panty check mam:; of the code is equivalent to a 
Founer matnx, i.e., the code is Rced-Solomon For a given m, the first code always requires fewer 
additions and multiplications, and for m =2, and 3, it also requires a smaller m, achieving the smallest 
field size possible. The simple, point-to-point technique is then extended to an arbitrary network 
topology. Implementation details of this technique in existing and future communication networks are 
discussed, and applications of the technique to trunk failures, short-term environmental disruptions such 
as fading channels in microwave radio networks or polarization dispersion in fiber optic networks with 
wavelength division multiplexing, dispersity routing in packet-switched networks, distributed storage, 
and fault-tolerant parallel transmission of continuous-amplitude discrete-time signals arc presented 


What is the Capacity of One Bit of Memory? 

Santosh S Vcnkatcsh Moore School of Electrical Engineering, University of Pennsylvania, 
Philadelphia, PA 19104 

We investigate the minmax problem of the maximization of the minimum amount of information 
that a single bit of memory retains about the entire past. Spccilically, we arc given a random binary 
sequence of ± inputs drawn from a sequence of symmetric Bernoulli Urals, and a family of (time 
dependent, deterministic or probabilistic) memory update rules which at each epoch produce a new bit (-1 
or 1) of memory depending solely on the epoch, the current input, and the current state of memory. The 
problem is to esumate the suprcmum over all possible sequences of update rules of the minimum 
information that the bit of memory at epoch n retains about the previous n inputs In this paper we show 
precise estimates and demonstrate that a sequence of probabilisuc memory updates-thc harmonic update 
ru/e-retains the maximum amount of information about the past We also mvesugatc natural 
generalizations of the problem when more than one bit of memory is available 


Minimum Bound of Auto- and Cross-Correlation of Sequences 

Shuo-Ycn Robert Li, and Ning Zhang Bellcore, Morristown, NJ 07960-1960 and Pacific Bell, 2600 
Cammo Ramon, IS900G San Ramon, CA 94583 

In some communications systems, a receiver uses matched filter detection for a desired signal. The 
detector outputs the correlauon function of the input signal with the desired signal. It is therefore 
preferable that the signal has an impulse-like autocorrelauon function. When multiple signals arc used in 
the same channel, it is also preferable that the crosscorrelauon function between two different signals is 
small in magnitude. We investigate the minimum bound on all crosscorrelaticns and all off-phase 
autocorrcllations of two signals, which are equally long sequences of unit vectors. In the special case 
when the two signals are generalized Barker sequences, we also investigate the minimum bound on all 
their crosscorrelations. 
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Binary Quadratic Form: a Solution to the Set Partitioning over GF(q) 

Celso de Almeida and R. Palazzo, Jr. Department ofTelematica, FEE-UNICAMP, P 0 Box 6101,13081 
Campinas, SP, Brazil 

In this paper it is presented a general soluuon to the set partitioning problem over GF(q) by use of 
the binary quadratic forms for bi-dimensionai lattices. From Fermat’s results. Genus and Composition 
Theorems, it is shown that when the solution is relatively pnme the resultant fundamental set form a 
Latin Square and that the least squared Euclidean distance between points belonging to the same coset 
is q. 


An Extended Cutoff Rate for Frequency-Hopping Communications with \on-Ideal Interleaving 
Shaul Laufer and Jakov Snyders Department of Electronic Systems, Tel-Aviv University, P 0 Box 39040, 
Tel-Avtv 69978, Israel 

The suitability of the capacity C and the cutoff rate R 0 , which are the customary measures for 
evaiuanng the channel's ability to transmit information reliably, turns out to be questionable with respect 
to frequency-hopping communications over channels with partial band jamming and non-ideal 
interleaving. More explicitly, non-ideal pseudo-random block interleaving is assumed, resulting in a 
block interference channel wit!, noise seventy level which depends on the number of jammed hops in 
each block of interleaving As the interleaving span increases, R a for this channel departs, inadequately, 
from the cutoff rate for the memorylcss channel Also, C does not reflect the behavior of the channel 
when short block codes, those usually encountered in interleaved commumcations, are used 

An extended cutoff rate R„ for channels with block memory is introduced. It is given by 

Ro'VWc) = nun j- j- log M E [aT' v - w 

where N, Nq, p, arc the interleaving span, codeword length and fraction of band jammed, respectively, 
R 0 {n) is the cutoff rate for a pscudo-randomly interleaved block with n jammed hops and the expectation 
is taken with respect to n Expressions for the extended cutoff rate are denved, both in presence and 
absence of side informauon. R„ has distinctive properties In particular, as the interleaving span tends to 
infinity, R„ converges asymptoucally and apparently monotomcally to R„ for the memorylcss case. 
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When Do Three Convex Corners Generate the Unit Simplex? 

K. Marton Mathematical Institute of the Hungarian Academy of Sciences. Budapest, FOB 127. H-1364 
Hungary 

Let Rt denote the non-negative orthant of the t-dimcnsiona! Euclidian space A set A c R{ is called 
convex comer if it is convex, compact, has nen-empty interior, and a e A,a'<a imply a' e A (''a' < a” 
is understood coordinatewisc.) The unit comer, or unit simplex, T is the simplex with vertices (0,.. ,0), 
(1,0,. ,0), (0,1,0,. ,0),...(0,0,...,1) The anublockcr of the convex comer A is the convex comer 

A' = lbeRi' (a,6) £ 1, all a e A] 

For two vectors x,y e Rj, x O y denotes the vector ( x,y, t i = 1, .it) It has Been shown recently that for 
two convex comers A and B, satisfying S c A \ we have 

T = A o B iffB = A’ 

In the present paper we give, under some restrictions, a necessary and sufficient condition for three 
convex comers A, B and D, to satisfy 


T = A o B0 D 


An Algorithm for the Piecewise Linear Approximation of Planar Curves 
Shuichi Itoh University of Electro-Communications, Chofu, Tokyo 182, Japan 

The approximation of two dimensional digital curve is treated within a framework of universal data 
compression The given curve is supposed to have random fluctuauons. Our object is to get a smooth 
approximation of it by a piecewise linear curve. We assume that the given curve is generated by a noisy 
observation of the desired approximation. According to this source model, those points on the given curve 
are encoded by first describing the approximaaon and then by describing the deviation of those points. 
Following Rissanen's Minimum Description Length Pnnciple, the approximation yielding the shortest 
description of those points is formulated. An algorithm is proposed to solve the above problem by a 
rcstncted optimization, whose computational complexity is of 0(N log N) where N is the number of the 
points on the given curve The algorithm is robust m a sense that no tolerance parameter is needed. 

Informatic Crossover in Genetic Algorithms 

Sami Khun Dept of Biomedical Eng, The Johns Hopkins University, Baltimore, Maryland 

Genetic algorithms are randomized (but not directionless) search procedures that maneuver through 
complex spaces looking for optimal solutions. They are based on the mechanics of natural selecuon and 
natural genetics. These procedures, which are implicitly parallel in structure, make few assumptions 
about the problem domain and can thus be applied to a broad range of problems 

A succession of generations created by reproduction, crossover and mutation constitutes a parallel 
search through the search space, favoring regions with above average fitness The crossover operation is 
of vital importance since a loss of diversity in a generation will generally yield premature convergence, 
pushing the search toward a sub-optimal solution. This paper presents "informatic crossover", which is 
based on Shannon’s inu.rmation theory It uses condiuona! entropy for selecting the crossing site in 
crossover and has been shown to promote diversity in generations 
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General Soft Decoding of Block and Convolutional Codes 

John T Coffey and Rodney M. Goodman California Ustitute of Technology, Pasadena, CA 91125 

We examine the complexity of general methods for decoding linear block codes with full hard 
decision decoding and (especially) bounded soft decision decoding. We compare the complexity of 
information set decoding with that of other algonthms, and show how improvements are possible in the 
case of bounded soft decision decoding Finally, we discuss the applicability of these results to the 
decoding of convolutional codes. 


On the Enumeration and Generation of Non Weight Equivalent Rate Vi Convolutional Codes 
Jean Conan and Chahm Fiouzi Departement de Genie Elearujue, Ecole Polytechnique de Montreal, P 0 
Box 6079, station "A", Montreal, Quebec H3C 3A7 

In this paper we investigate the question of weight equivalence of rate Vi binary convolutional codes 
and consider the problem of enumeration and generation of all the non weight equivalent such codes for a 
given memory order m. In the generation process, this will be done by ordering the potential encoders 
into classes in such a way that it becomes easy to recognize the weight equivalence as well as the 
catastrophic error propagation condiuons. Consequently, sub-classes including self and non self 
reciprocal as well as catastrophic and non catastrophic such encoders will be introduced. The cardinal 
number of these ensembles as well as the number of "non weight equivalent" codes will be computed 
recursively as a function of m Since these relations amount to simple convolutions, they can be 
compactly represented by generating functions which wiU also be presented 


Convolutional Codes for Finite State Channels 

Manju Hcgde, Mort Naraghi-Pour, and Xiaowci Chen Dept of Electrical and Computer Engineering, 
Louisana State University, Baton Rouge, LA 70803 

Error control strategics for finite state channels often utilize decoders designed for use over 
memoryless channels ("memorylcss decoders"). These are used with either specifically designed codes 
such as burst error correcting codes or with landom error correcting codes in conjuncuon wnh 
interleaving. It may be possible, however, to improve the error probability performance of coding 
schemes by employing decoders which exploit 'he memory in the channel. Such decoders would utilize, 
directly or indirectly, the channel state sequence information contained in the channel output sequence. 

The notion of state in convolutional codes and the existence of efficient decoding algorithms which 
are sequential (e g., Vitcrbi algorithm) make convolutional codes suitable for adaptation for use with 
decoders which utilize the memory in the finite state channel without greatly increasing the concomitant 
complexity. In this paper we propose several new decoders for the use of convolutional codes over such 
channels The main feature of these decoders is that they can be implemented sequentially on a trellis by a 
"Viterbi-hke" algonthm. We evaluate the performance of these decoders by simulation and compare it to 
the pcrfoTnance of "memorylcss decoders" with and without interleaving Our results indicate that in 
cases where the channel is bursty, these decoders significantly outperform the "mcmoryless decoders" 
The cost of this improved performance is somewhat moderate increase m the decoders complexity 
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A Method for Calculating Weight Distribution of ft =kln Convolutional Codes 

Hiroshi Sasano -nd Masao Kasahara Faculty of Science and Technology, Kinki University, Higashi- 

osaka, Osaka 577, Japan, and Department of Electronics, Kyoto Institute of Technology, 

ConvoluUonal codes are nowadays extensively used on the vanous communication systems. In 
order to estimate the performance of convolutional codes, weight distnbuuon is one of the most important 
parameters. Unfortunately, for rate R = kin codes, it is not easy to obtain the weight distnbuuon even 
with relatively short constraint length. 

In this paper, an efficient method for calculating we'ght distribution of binary rate R=kln 
convolunonal codes is presented It extends the earlier work for ft = 1/n convoluuonal codes. The method 
is based on the idea of the vanation of Hamming weight of codewords caused by input binary sequences 
of an encoder, and the rule of the variation defined by a vanation and the input sequence which 
determines it. Weight of a codeword is expressed by the sum of the variations and the base-weight which 
is the weight of the codeword corresponding to the shortest input sequence The weight distnbuuon is 
obtained iteratively m order of weight of codewords. This method consists of several procedures It is 
shown that the procedures are systematic and easy to be handled 


The Free Distance of Fixed Convolutional Rate 2/4 Codes Meets the Costello Bound 
V. V Chcpoyzov, B J. M. Smeets, and K. Sh Zigangtrov Inst for Problems of Information 
Transmission, USSR Academy of Sr ences, Moscow, USSR, Dept of Information Theory, University of 
Lund, P 0 Box 118, S-221 00 Lund, Sweden, and Inst for Problems of Information Transmission, USSR 
Academy of Sciences, Moscow, USSR 

The free distance df, a of fixed convoluuonal codes of rate ft = 2/4, is asymptotically lower 
bounded by the Costello bound on df r „ for time varying convolutional codes of tins rate. In particular, we 

can show that for these fixed codes we have -^~=P c -0 (—-—), where u=4(m + l) is the 

t> ' logfu) 

constraint length, m is the memory, andp c = - ft/log(2‘~ K - 1) = 0 3932 is the Costello parameter for rate 
ft = 2/4 codes. 


Design of Non-Systematic 3-SyEC/AUED Codes of Asymptotically Optimal Order 
Sandip Kundu IBM TJ. Watson Research Center, 2-1-258, P 0 Box 218, Yorktown Heights, NY 10598 
This paper considers the design of balanced binary block codes that arc capable of correcting up to 3 
symmetne errors and detecting all unidirectional errors. Non-systcmalic 3-Symmctnc- Error- 
Correcung/All-Undircctional-Error-Detccting (3-SyEC/AUEQ) codes constructed tn this paper arc of 
asymptotically optimal order in redundancy Little has been published on non-systcmatic t-Symmetnc 
Error Correcting/AU-Undirectional-Error-Detecting codes. 1-SyEC/AUED codes were constructed in 
1977 and 1981,2-SyEC/AUED codes in 1984 and 1988 [by the author] 


Constructions and Bounds for Systematic and Nonsystcrnatic rEC/AUED Codes 

Frank J. H. Boinck and Hcnk C. A van Tilborg Department of Mathematics and Computing Science, 

Eindhoven University of Technology, Eindhoven, The Netherlands 

Several methods to construct systematic and nonsystcrnatic r-error correcting/all unidirectional 
error-detecung codes are described. The construction of nonsystcmauc lEC/AUED codes, presented here, 
makes use of existing constant weight codes or of block designs. 

Systematic rEC/AUED codes can be made by adding a tail to a linear (-error correcting code, but 
other constructions are of an ad hoc nature. They will often be found as suitably chosen subsets of 
nonsystematic rEC/AUED codes. 
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Further bounds and extensive tables on the word length of systematic and nonsystematic 
rEC/AUED codes are presented. 


A New Technique for Constructing t-ECld-EDIAUED Codes 

R Venkatesan and Sulaiman Al-Bassam Faculty of Engineering and Applied Science, Memorial 
University of Newfoundland, St. John’s Canada AJB 3X5 and Deparonent of Computer Science, Oregon 
State University, Corvallis, OR 97331 

This paper presents a procedure to construct r-error correcting/ d-error detecting/all unidirectional 
error detecung ( t-ECld-EDIAUED ) codes. This technique is similar to those recently reported to 
construct t-ECIAUED codes. A t-ECId-ED code is chosen and then a tail is appended in such a way 
that the new code can detect more than d errors when they are unidirectional. The efficiency of the 
resulting t-ECld-EDIAUED codes compares favorably to the existing method. In addition, this 
procedure enables simple and fast encoding and decoding algorithms. An upper bound for the number of 
additional check bits required to incorporate unidirectional error detection capability is denved. 
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SESSION TA7 


TRELLIS CODING II 


Trellis Precoding 

M. Vedat Eyuboglu and G. David Fomey, Jr Codex Corporation, 20 Cabot Boulevard, Mansfield, MA 
02048 (40 min.) 

On a Gaussian channel with intersymbol interference and/or colored noise, trellis precoding is a 
method of combining the equalization performance of an ideal decision-feedback equalizer with the 
coding gam of known lattice-type coset codes, and in addition with substantial shaping gain. Trellis 
precodmg is a generalization of trellis shaping to non-ideal channels, on the one hand, or of Tomlinson 
precodmg to coded systems, on the other. In principle, on any linear Gaussian channel, trellis precoding 
can be used to approach the Shannon limit. The method is quite practical whenever channel information 
is available to the transmitter. 


Erasure-Free Sequential Decoding and Its Application to Trellis Codes 

Fu-Quan Wang and Daniel J. Costello, Jr. Department of Electrical and Computer Engineering, 
University of Notre Dame, Notre Dame, IN 46556 

The Erasure-Free Fano Algonthm (EFA), a Fano algorithm version of the Multiple Stack Algonthm 
(MSA) which is erasure-free in the sense that the buffer is guaranteed not to overflow if a large speed 
factor is used, is presented Simulation results show that the EFA can perfoim as well as Ihe MSA but 
requires a larger computauonal effort Although the speed factor required to guarantee erasure-free 
decoding is large, the average computauonal load for both the EFA and the MSA is small. Then a 
modified version of the EFA, called the Buffer Looking Algonthm (BLA), is investigated Simulation 
results show that the BLA needs a much smaller speed factor to guarantee erasure-free decoding than the 
EFA or the MSA while good performance is retained. Ihe performance, complexity, and delay of the 
BLA arc compared to the Vitcrbi algonthm. The BLA is shown to achieve equal or lower error 
probabihues wilh reduced complexity but a larger delay The application of the BLA to the decoding of 
convolutional and trellis codes is discussed (This work was supported by NASA grant NAG 5-557 and 
by NSF grant NCR 89-03429) 


Entropy-Constrained Trellis Coded Quantization 

Thomas R. Fischer and Min Wang Department of Electrical and Computer Engineering, Washington 
State University, Pullman, WA 99164, and Department of Electrical Engineering, Texas A&M University, 
College Station, TX 77843 

Entropy-constrained trellis coded quantization (ECTCQ) combines entropy coding with flic trellis 
coded quantization (TCQ), yielding an encoding scheme that offers improved performance over entropy- 
constrained scalar quantization with only a modest increase in encoding complexity. For mcmorylcss 
Gaussian and Laplacian sources, the ECTCQ system with 8-statc trellis provides mean-square error 
(MSE) performance within about 0.6 dB of the respective distortion-rate function for encoding rates 
above about 1.5 bit/sample Sources with memory can be encoded with a predictive ECTCQ system For 
Gauss-Markov sources and encoding rates above 1 5 bit/sample, predictive ECTCQ with an 8-statc trellis 
yields MSE within about 0.7 dB of the respective distortion-rate function. Good performance at lower 
encoding rates is achieved by using vector codebooks in the ECTCQ system 
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The Design of Joint Source/Channei Trellis Coded Quantization/Modulation 
Min Wang and Thomas R. Fischer Department of Electrical Engineering, Texas A&M University, 
College Station, TX 77843 and Department of Electrical and Computer Engineering, Washington State 
University, Pullman, WA 99164 

A joint trellis coded quantization (TCQ) ard trellis coded modulation (TCM) system is designed by 
selecting identical trellises in the TCQ and TCM, and mapping the quantization levels to modulation 
symbols. An algorithm is derived for the joint optimization of the TCQ levels and TCM symbols. 
Extensions of the approach allow predictive encoding and multidimensional codewords and modulation 
symbols. 


Regular Labelings for Trellis Codes with Rectangular Signal Constellations 
Ying Li Dept. ofEECS, MIT, Cambridge, MA 02139 

Regular trellis codes have the desirable property that the the minimum distance is relatively easy to 
calculate, and thus uie code search and the performance evaluation are simplified. This paper presents 
new results on regular trellis codes using rectangular signal constellations. A "labeling” for a trellis code 
is a mapping from coded bits to subsets of signals in the partitioned signal constellation. A ’’regular 
labeling" is required for a trellis code to be regular. We study structures of regular labelings for 
rectangular signal constellations where the subset partitioning does not necessarily correspond to cosct 
decomposinon. It is shown that for an m-dimensional constellation partitioned into more than 2 2 " 1 
subsets, regular labelings, and thus regular binary trellis codes, do not exist. For an m-dimcnsional 2 2m - 
way partiuon, it is shown that there is a unique structure for regular labelings. For m-dimcnsional 
partiuons of fewer than 2 1 " subsets, new structures for regular labeungs arc found. 


Performance Bounds for Trellis Coded Direct Sequence Spread Spectrum Multiple Access 
Communications Systems 

Brian D Woemer and Wayne E. Stark The Department of Electrical Engineering and Computer Science, 
University of Michigan, Ann Arbor, MI 48109 

This paper considers a Direct Sequence Spread Spectrum (DS/SS) multiple access communications 
system. One may view a binary antipodal signature sequence as a low rate rcpeuuon code. Other choices 
for sets of signature sequences, corresponding to different codes, are possible 

We have previously proposed the use of trellis coded signature sequences as a means of improving 
the average probability of bit error for this type of communication system. The trellis codes are obtained 
by performing set partitioning on a set of biorthogonal signature sequences. These codes have a coding 
gain over several decibels over the corresponding binary antipodal commumcauons system. 

We assume that the communications system employs a correlation receiver and a maximum 
likelihood (Vitcrbi) decoder. We apply a technique onginally introduced by Lchnert and Pursley to 
analyze the performance of a DS/SS communication system with binary antipodal signature sequences. 
By slightly modifying this technique, we arc able to obtain accurate performance results for our trellis 
coded systems. The results show that our trellis coded systems exhibit performance supenor to that of 
binary antipodal DS/SS multiple access systems. 


New Trellis Codes over GFIQ) for One and Two Dimensional Lattices 

Celso de Almeida and R. Palazzo, Jr Department of Telematics, FEE-UNICAMP, P 0. Box 6101,13081 
Campinas, SP, Bratil 

In this paper new trellis codes over GF(q) are presented for one and two dimensional Euclidean 
spaces, it is denved a closed form expression for the squared free Euclidean distance and for the 
asymptotic coding gam for the combined form of M-PAM with convolutional codes with one and two 
memory elements. The corresponding generator matrices for a class of these codes are also presented. 
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TECHNICAL SESSIONS 


Tuesday, 2 p.m. - 5 p.m. 


SESSION TPI 

STOCHASTIC PROCESSES 


Extension of Slepian’s Model of Gaussian Noise 

Nelson M. Blachman GTE Government Systems Corp, Mountain View, CA 94039-7188 

The notion underlying the Schmidt orthogonalization of polynomials can be applied to finite linear 
combinations of distinct variables. These variables may represent the value x(t) and denvauves tl (r), 
x"(r), , x m (t) of a zero-mean stationary random process at a given instant For this application the 

properties of the least-mean-square linear predictor of the random process are readily developed. The 
N +1 terms of this predictor are linear functions of x(t) and increasing numbers of its denvauves, they 
are orthogonal to each other as well as to the error By specializing to the Gaussian case, tire Slepian 
model - the sum of the first two terms plus us error -- is easily extended. The N +1 terms are 
determimsne functions of time that give the model the nght conditional mean, and the (N +2)nd term (the 
corresponding error) is a zero-mean nonsrauonary Gaussian process that gives the model the proper 
conditional covariance. The details for a process with a Gaussian spectrum illustrate the results 


Sampling Designs for Estimating Integrals of Stochastic Processes 

Karim Bcnhennt and Stamatis Cambams Department of Statistics, University of North Carolina, Chapel 
Hill, North Carolina 

The problem of estimating the integral of a stochastic process from observations at a finite number 
of sampling points is considered. Sacks and Ylvisakcr (1966, 1968, 1970) found a sequence of 
asymptotically opumal sampling designs for general processes with exactly 0 to 1 quadratic mean (q m ) 
denvauves using optimal-coefficient esumators, which depend on the process covanancc These results 
were extended to a rcstneted class of processes with exactly K q m derivatives lor all K =0, 1, 2, , by 
Eubank, Smith and Smith (1982). The asymptouc performance of these optimal-coefficient estimators is 
determined here for regular sequences of sampling designs and general processes with exactly K q m 
denvauves, K> 0 More significantly, simple nonparametne esumators based on an adjusted trapezoidal 
rule using regular sampling designs are introduced whose asymptouc performance is idenucal to that of 
the opumal-cocfficicnt esumators for general processes with exactly K quadratic-mean denvauves for all 
AT = 0,1,2, .. 


Almost Sure Convergence of Autoregressive Spectral Estimation 

Elias Masry Department of Electrical and Computer Engineering, University of California at San Diego, 
La Jolla, CA 92093 

Let (Y,)"— be a stauonary autoregressive process of order p with AR-paramctcis i,, 
innovation vanance a], and spectral density tp(X). The observauon process is (F„=V„+H / „)”„, where 
is an addiuve white noise. On the basis of a finite length noisy observation {T / ,)J , ,i, 
appropnate estimates of the parameters {a ; )f.i, a\, and the spectral density 0(A) arc considered and 
their almost sure convergence properties are denved. Sharp rates of almost sure convergence of these 
estimates.are established (strong laws of the iterated loganthm) In particular, for the spectral density 
estimate # W (A,), we have uniformly in L, that 
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almost surely as N -» where u N -*°° as A' ~ (at an arbitrarily slow rate). 


Level-Crossing Analysis by Means of a Scaling-Dimensionality Transform 

A. Baibd Department of Electrical Engineering, Katholieke Universiteit Leuven, K Mercierlaan 94, B- 
3030 Heverlee, Belgium 

It is well-known that the application of S.O. Rice’s formula for the average number of level 
crossings over a certain time interval, gives the theoretically senseful but practically nonuseful result of 
this number being infinite in case the process considered is nondtfferentiable The problem in this 
contribution deals with a practicable characterization of the mean level-crossing activity of stanonary 
processes. 

With E N a (6) denoting the expected number of crossings of a fixed level a pet unit of time, when 
the underlying process x(t) is sampled and linearly interpolated with sampling penod 0, the following 
function is defined: D„(r,0) = [log EN a (6)-\ogEN a (rQ)}l]ogr. It is called the scaling-dimensionality 
transform and is somewhat related to the concept of fractal dimension. First, the properties of this 
transform are discussed in a general context. Then results will be given for stationary Gaussian processes 
for which the EN a (6) arc easily calculable starting from Rice’s formula. D a {r,0) is always finite, even 
for0->O, as opposed to EN a (0). For 0 -> 0, DJr.O) corresponds to the fractal dimension of a related 
fractional Brownian motion as earlier considered by B. Mandelbrot. The concept of signal blumng is 
introduced, and its influence on the D a (r,0)-transform is analyzed This allows for a well-argued 
assessment of the level crossing activity of these signals, resulting in a well-balanced choice of both 
amplitude- and omc-scale (sampling interval) for pixel-based signal representations 


The Curve Crossing Problem of a Gaussian Random Process 

T. Munakata, T M-rnaki, and D.Wolf Faculty of Engineering, Tamagawa University, Machida, Tokyo, 
Japan, University of Electro-Communications, Chofu, Tokyo, Japan, and Institut fuer Angewandte 
Physik, University of Frankfurt aM ,F RG 

For a Gaussian random process having the autocorrelation function m (r), the Rice function Q ± (,) of 
the curve crossing problems is reduced to a stmplc expression, if the curve has the form 
C (r) = m(z)a + g(T), where a denotes the start level and g(t) an arbitrary funcuon This expression 
Q ± () does not contain the value a explicitly, ft means that the Rice function Q ± () is invariant for all 
curves defined above with given g(r) and arbitrary value of a. Furthermore, the following two special 
cases arc very interesting. 

CASE 1 • for g (t) = 0, i e , C (T) = m (t )a 

The expression £!*() becomes equal to the Rice function for the zero-crossing problem Tins suggests 
that for the Gaussian process the property of curve-crossing intervals is invariant for any C (r) = m (t)a 
CASE 2. forC(T) = m(r)a + (l -m(t))b 

Q ± (:') becomes equal to that for the level-crossing with level b Now, one may question whether a similar 
discussion can be applied to the interval lengths between adjacent curve-crossing points. Since the curve 
crossing problem is not solved analytically,, we made a computer simulation to answer the above 
question 
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Multidimensional Random Fields Equivalent to Time Processes 

Millu Rosenblatt-Roth Department of Electrical Engineering, The City College of the City University of 
New York, New York, NY 10031 

The paper presents some main aspects of the theory of Markov random meshes' (a) The formal 
definiuon of the concept, (b) Because a Markov random mesh is a Maikov field, it is necessary to express 
the characterizing elements of the later one as functions of the characterizing elements of the previous 
one. (c) Properties of the concept (d) The homogeneous case, (e) The one-dimensional representation, (f) 
Examples of Maikov meshes and of the corresponding Markov fields (g) Examples of Maikov fields 
which are not Markov meshes. The first example of such a random field appeared m 1965 (K. Abend, T.J. 
Harley, L.N. Kanal), but this idea was not pursued and quite forgotten. The aulhor of this Abstract 
developed it to the level we can use it for modeling by best approximation of arbitrary random fields with 
the help of Markov meshes. 


Constructing IVP Models with Specified Behavior on Certain Tail Events and Cylinder Sets 

Amir Sadrolhefazi and Terrence Fine School of Electrical Engineering, Cornell University, Ithaca, NY 

148S3 

We consider ume-sencs models that can incorporate the seemingly contradictory properties of 
stationanty, continuity, and support of bounded yet divergent time averages While the standard ergodic 
theorems rule out the existence of such models in the context of addiuvc probability measures, there exist 
models with these properties in the framework of Interval-Valued-Probability (IVP) Formally, IVP is a 
pair of nonnegative and normalized set functions (P,P), defined on a measurable space (OMEGA,A), 
with P superadditive, P subadditive, and satisfymg P( A) + P(A c ) = 1. We take our measurable space 
(OMEGA, A) as fl= {0,1 } N , A =<r(C), where C is the field of all cylinders on U Let D* denote the 
divergence event (binary sequences with divergent averages), and define the dimension of a cylinder C,. 
dim(C), as the minimum number of coordinates required to specify C We arc interested m IVP models 
that can incorporate the following. 

1) Stauonarity 

2) Continuity along C : P(u M C,) = lim„P(u"»i C,) and / > (o, 21 C,) = lim„/ > (n; , . 1 C.) where C, e C. 

3) Support for divergence: P(D*) = P(D *) = 1 

4) Marginal constraints: Given an additive measure p and a fixed integer k, P(C)-fi(C) = P(C) for 
every cylinder C with dim (C) < < 

Specific constructions of IVP models that satisfy 1-3 have been given by Papamarcou and Gnzc, 
however, these constructions do not address the marginal constraints We proceed to show that modulo 
the continuity condition, the issues of support for divergence and marginal constraints are unrelated for 
stationary IVP models We provide sufficient conditions for the construction of IVP models that satisfy 
1-4, and give specific instances where these sufficient conditions are satisfied 


Towards Robust Bispectrum Estimates 

David J. Thomson AT&T Bell Laboratories, Murray Hill, New Jersey 07974 

In many applicauons where the use of higher-order spectra is desirable, use of conventional 
methods is hampered by lack of data or, equivalently, by the evolutionary nature of the process. Esumatcs 
of bispectra are known to be highly vanable and to exhibit anticonsistcnt behaviour. Here we study a 
robust variant of a multiple-window method for computing consistent estimates of the bispcctrum from a 
short segment of the process. Bispectrum estimates are formed by robust time averages of triple products 
of what are effectively complex demodulates centered at f\,ft, and -f\ -fo 
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SESSION TP2 


MULTIPLE ACCESS I 


Multiple-Access Coding with Error Control: A Code Construction for the Real Adder Channel 
Peter Mathys Department of ECE, Box 425, University of Colorado, Boulder, CO 80309 

A concatenated code construction for a noisy multiple-access channel (MAC) is given. The MAC 
which is considered is the binary input discrete-time real adder channel (DTRAC) with unknown gains, 
an unknown offset, and additive uncorrelated noise. It is assumed that the DTRAC is shared by a large 
number N of users of which at most T are active simultaneously. A nonlinear binary blockcode which is a 
subcode of a linear cyclic code (and thus easy to decode) over GF(p£), Pe > 2, is used for error control 
purposes Linear binary codes of blocklength N are used for multiple-access coding. The latter codes have 
the property that the codewords of individual users which arc active simultaneously can be separated in 
the generalized frequency domain over GF(ph). 


Coding for the Multiple Access Channel with Sum-Rate Constraint 

Bixio Rimoldi Electrical Engineering Department, Washington University, St. Louis, MO 63130 

Consider M independent users sharing a discrete-time adder channel where the sum is taken over a 
finite field GF(q). User i, i = 1, ,.,M, is allowed to transmit at the maximal rate R 1 , but might choose to 
transmit at an effective rale R{, 0 £ R‘ E 5 R‘. The problem is to find a linear block code C‘ for each user r, 
such that the rate requirements arc met and such that the transmitted information words can be recovered 
by the receivers provided that the (effective) sum-rate R, = R‘ E satisfies R, where R m „ is a 

fixed design parameter. Coding for both noiseless and noisy chaiincls arc discussed. A pracuca] situation 
leading to the considered channel is encountered when M users want to transmit information across a 
single user channel with q-ary input and 17 -ary output, in this case an adder over GF (q) can be inserted in 
front of the channel without decreasing the channel capacity. 


Code Constructions for Asynchronous Random Multiple-Access to the Adder Channel 

Eli Plotruk Department of Electrical Engineering, Techmon - Israel Institute of Technology, Haifa 

32000, Israel 

We consider a situation where up to T randomly chosen users, out of M potential ones, 
simultaneously transmit data over the noiseless Adder Channel. These T (or less) aefive users operate 
independently, and because of unknown time offsets among their clocks, and different delays tha> me 
various messages incur dunng transmission, both block and bit synchronism arc precluded. Code 
constructions that ensure error-free asynchronous communication over this channel arc presented. The 
information rate of these codes is [T(l + A/m )) -1 per user, where A is the maximal delay (measured in 
bits) between the transmission and reception times of a message in the system, and m is a free design 
parameter. If exactly T users are active, use of these codes leads tc a stable throughput arbitrarily close to 
1 message/slot Furthermore, if the occurrence of collisions is made known to the active transmitters, 
such a throughput can be maintained for arbitrary T, T S M by means of an adaptive use of the codes 
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Channel Capacity of Multiple-Access Channel with Binary Output 

Yoichiro Watanabe Stanford University, on leave from the Department of Electronics, Doshisha 
University, Kyoto, 602 Japan 

A sufficient condition for the channel capacity is proposed for a multiple-access channel with binary 
output. It is proved that a local maximum point of the mutual information gives the channel capacity. 
Since the mutual information of the multiple-access channel takes the value on the non-convex domain, 
the Kuhn-Tucker conditions concerning the mutual information is necessary conditions which determine 
the channel capacity. This is contrasted for the fact that the Kuhn-tucker conditions for the ordinary 
discrete memoryless channel are the necessary and sufficient conditions of determining the channel 
capacity. Thus the sufficient condition is desired to exactly determine the channel capacity. The channel 
capacity is indispensable to evaluation of the capacity region for a multiple-access channel. 


The Capacity Region of the Random-Multiple Acces , Channel 

Eli Plotnik Department of Electrical Engineering, Techmon - Israel Institute of Technology, Haifa 
32000, Israel 

The Random-Multiple Access Channel (R-MAQ is a communication system with M potential 
users, of which, T users or less are selected at random to transmit their messages simultaneously, over a 
common discrete memorylcss channel, these T or fewer users are referred to as the active users, and they 
are fully synchronized. Based on the channel output vector, the decoder has to identify how many and 
which users are active, as well as their transmitted messages, with negligible probability of error. The 
capacity region of the R-MAC is defined as the set of all rate vectors (R u R 2 . , R«)e that ensure 
reliable communication. 

Denote by X t (1 = 1,2,.., M) the alphabet of user / and by T the channel's output alphabet and let 
St (l- 1,2, .., T) be the set of all subsets with exactly / users When the users in the subset s e S ( are 
acuve, the channel is characterized by the transiuon probability measure E(y(ls) We prove the 
following theorem that establishes the capacity region of the R-MAC 
Theorem 

Let P(X 1 ,^ 2 , ..,X U )^P i(X]) (P 2 (X 2 ) Pm(X m ) be a probability distribution, and denote by 
T(P\ Pi ,P M ) the set of points (R \ R 2 ... IR" that satisfy the following conditions for every 
/./el, 2. M(I*J)-. 

Ri<min[l(Xi,Y) mm mm 
r«U. ,r-i js s, 
las 


Differentially Coherent Multiuser Detection in Code-Division Multiple-Access Channels 

Mahesh K. Varanasi Department of Electrical and Computer Engineering, University of Colorado, 
Boulder. CO R0309 

The demodulation of differentially phase-shift keyed signals transmitted simultaneously via a 
CDMA channel is studied under the assumption of white Gaussian background noise. A class of bilinear 
detectors is defined with the objective of choosing the optimal bilinear detector. The optimality cntcrion 
considered is near-far resistance which denotes worst case bit error rate in low background noise over 
near-far environments. The optimal bilinear detector is therefore obtained by solving a minimax 
optimization problem and is shown to be a decorrelating detector. The bit error rate of the decorrelator for 
each user is equivalent to that of the corresponding optimum single-user DPSK detector in a reduced 
energy single user environment. A useful property of the decorrelator therefore is that the bit-error rate of 
each user is invariant to interfering signal energies and phases, thereby eliminating the near-far problem 
associated with the conventional single-user detection scheme. It is further shown that no other DPSK 
multiuser detector has a higher near-far resistance than does the decorrelator. That is, the optimally near- 
far resistant bilinear detector is optimally near-far resistant among all DPSK detectors. 
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Capacity of RMS Bandlimited Gaussian Multiple-Access Channels 

Roger S. Cheng and Sergio Veidii Department of Electrical Engineering. Princeton University, 
Princeton, NJ 08544 

Continuous-tine additive white Gaussian noise channels with strictly time-limited and root mean 
square (RMS) bandlimited inputs are studied. RMS bandwidth is equal to the normalized second moment 
of the specttum, which has proved to be a useful and analytically tractable measure of the bandwidth of 
strictly time-limited waveforms. 

We show how the classical formulas for the capacity of strictly bandlimited single and two-user 
channels change under the RMS bandwidth measure. In addition, we consider channels where the inputs 
are further constrained to be Pulse Amplitude Modulated (PAM) waveforms. We analyze those channels 
under the assumption that the transmitters are symbol-synchronous and we find the pair of pulses that 
achieves the boundary of the capacity region. The shapes of the optimal pulses depend not only on the 
bandwidth but also on the respective signal-to-noise ratios. 


The Effect of Coding on the Reliability of Computer Memories 

Rajeev Krishnamoorthy and Chris Heegard School of Electrical Engineering, Cornell University, Ithaca, 
NY 14853 

We consider the problem of the Mean Time to Failure (MTTF) of a random access memory which 
is protected by an on-chip single-error-correcting code. In a previous paper we studied the 
implementation of a code on a memory array and analyzed the effect on the yield (the probability that 
every bit on the memory array can be read correctly.) We restrict our attention to single-cell failures, 
since the purpose of coding is to combat single-cell defects. 

We assume that cell failures form a Poisson process with parameter A and that cell failures in a 
codeword form a Poisson process of intensity «A. We take into account the event that not all codewords 
on the memory array arc defect-free 

We derive an expression for the MTTF of a coded memory army and analyze its behavior as p 
vanes, where p is the probability that a cell is defective at t = 0. As p -> 0 the MTTF of coded arrays is 
proportional to AT 14 (where N is the size of the memory), compared to AT 1 for the uncodcd case. Coding 
therefore provides two benefits: it significantly increases the lifetimes of the mcmoncs, and slows down 
the rate of decrease of the MTTF as N increases 
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SESSION TP3 
SIGNAL PROCESSING I 


Predictive Contour Coding for an Object-Oriented Analysis-Synthesis Coder 

M. Hotter and K. W. Hahn Institut fur Theoretische Nachrichtentechnik und Informationsverarbeitung, 

Universitdt Hannover, Appelstrasse 9A, 3 Hannover 1, FJt G. 

A predictive contour coding algorithm is presented which encodes the temporal differences of 
object shapes as generated in an object-oriented analysis-synthesis coder for moving images. Using stored 
shape information from the previous image, the object shape is approximated by a combination of 
polygon- and spline-representation. The quality of the contour approximation depends on the number of 
vertices which is adapted to the available transmission capacity, i.e., the smaller the data volume the 
coarser the approximation. By the presented algorithm, a high quality contour approximation can be 
achieved with an average bit rate of only about 0.3 bit per contour pixel. Therefore the presented 
predictive contour coding algorithm is appropriate for object-oriented coding of moving images at very 
low transmission rates where an efficient description and coding of the object shapes is needed. 


The Extended Berlekamp-Massey Algorithm 

Willard L. Eastman The Mitre Corporation, Burlington Road, Bedford, MA 01730 

The Berlekamp-Massey algorithm solves a linear Hankel system of equations with a special nght- 
hand-side. Since a Tocplitz system of equations can be reordered to convert it into a Hankel system, the 
algonthm can also be used to solve a Tocplitz system with a special nght-hand-sidc. Tocplitz systems of 
equations arise in a number of important signal processing applications, but for some applications it is 
necessary to solve a system with an arbitrary right-hand-side. In this talk we extend the Berlekamp- 
Massey algorithm for synthesizing the shortest-length linear feedforward combinational circuit (LFCC) 
that generates a second given sequence t from the contents of the shortest-length linear feedback shift 
register (LFSR) generating a given sequence s. The LFCC synthesis algorithm can solve p xp linear 
Hankel orToeplitz systems of equations with arbitrary nght-hand-sides. Efficient VLSI implementations 
of algorithms for solving general Tocplitz systems are of great interest. Such an implementation is 
presented. 


Complex Sequences over GF(p M ) with a Two-Level Autocorrelation Function 
M Antweiler and L. Bomer Institute for Communication Engineering, Technical University of Aachen, 
D-5100 Aachen, West Germany 

The paper proposes new complex sequences with elements which have constant absolute values of 
1. The pcnodic autocorrelation functions of these sequences are shown to be two-level. Such sequences 
play an important role in synchronization, radar applications or in code division multiplex systems The 
sequences are generated by three consecutive mapping processes: the first one maps the elements of 
GF(p M ), (p prime; M integer) into the elements of GF(p J ), (J integer and a divisor of M)\ the second one 
maps the elements of GFip 1 ) into the elements of GF ip). Finally the elements of GF(p), which are 
integers in the range (0,/7-lJ, are transformed into the pth roots of umty which build the complex 
sequence. The first and the second mapping steps are expressed in teims of the well known trace function. 
The linear span of the new sequences is examined and is proven to be larger than the linear span of 
complex m-sequences, if the parameters of the mapping processes are appropriately chosen. One 
realization of a sequence generator is proposed, consisting of shift registers, adders and a read-only 
memory (ROM). 
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Construction of a New Class of Higher-Dimensional Legendre- and Pseudonoise-Arrays 

L. Bomer and M. Antweiler Institute for Communication Engineering, Aachen University of Technology, 

Aachen, West Germany 

In this paper a new construction method for synthesizing two- and higher-dimensional Legendre 
arrays is introduced. The resulting arrays are 4 -dimensional arrays and every dimension has the size p, 
where p denotes an odd prime. The first element of the ^-dimensional array is of value ”0". All other 
elements are of value ” -1" or ” +1". With a theorem it is shown that all sidelobes of the penodic 
autocorrelation function of these arrays are " -1" If q is set to 1, the well known Legendre sequences arc 
constructed. 

If the first element of these arrays are replaced by " -1” or " +1”, a new class of binary arrays is 
generated. It is shown that these arrays are pseudonoise arrays, if the number of elements N equals 3 mod 
4. For N equals 1 mod 4 the penodic autocorrelation function sidelobes are either + -1" or" +3". 


Multiple Bases Signal Representation, Coding, and Reconstruction 

A A. (Louis) Beex and Felix G. Safar Bradley Department of Electrical Engineering, Virginia 
Polytechnic Institute & State University, Blacksburg, VA 24061 

The problem of efficient signal communication at low data rates involves, in general, the encoding 
of the source for maximum data compression at the transmitter end, and the reconstruction at the receiver 
end, from the received information and all the available a prion 01 side information. We propose an 
adapnve signal representation scheme, based on the use of multiple orthogonal basis sets, that exhibits 
very good potential in beth the source encoding and the signal rcconstrucuon end of the above problem. 
Our representauon leads naturally to the splitting of the signal into additive components, each of which 
has a simpler description than the original process In addition, 11 exhibits a structure similar to that of 
codebook based coding. As a result, a very compact signal representation can be achieved, A splitting 
procedure called recursive residual projection is proposed, and its performance evaluated for the 
separation of imagelike line signals into basis-defined "edge" and "texture" components. The coding of 
these components leads to lower rates than those for transform coding methods In reconstruction, the 
representation can be considered as a well-behaved constraint This allowed for the development of the 
corresponding unique projection operator, applicable for iterative reconstruction methods in general. In 
particular, we also propose a noise tolerant version of this operator, a so-called soft projection operator, 
capable of achieving convergence under noisy measurement conditions Computer simulations of the 
representation, coding, and reconstruction aspects corroborate the usefulness of this proposed 
representation. 


Fast Vector Quantization Algorithm by Using an Adaptive Search Technique 

Kohji Motoishi and Takesi Misumi Department of Computer Science and Communication Engineering, 
Kyushu University, Fukuoka 812, Japan 

This paper presents a fast vector-quantizauon algorithm that yields an optimal code at a small 
number of searches by using an adaptive search technique. In preparation, we made a table of distances 
between codes and decide the first probing code on the basis of a given codebook. When an input vector 
enters, the distance between the input vector and the first probing code is calculated. Comparing the 
calculated distance with the distance table, some codes are automatically eliminated from candidates for 
the optimal code. The efficiency of the algonthm, that is, the number of eliminated codes, is greatly 
influenced by how to select the probing code. The first probing codes is decided in advance on the basis 
of statistical information on the training sequence. The second and following probing codes are 
adaptively selected so that the number of eliminated codes becomes larger. The result of a preliminary 
experiment showed that the number of search was reduced to 1/11.7 in companson with the full search 
algorithm and to 1 / 2.12 in comparison with the tree search algonthm. 




An Analysis of Cepstrum Via Wave Moments 

Anil Khare and Toshinori Yoshikawa Department of EE System Engg, Nagaoka University of 
Technology, Kamitomioka-Machi, 1603-1, Nagaoka-Shi, Niigata-ken 940-21, Japan 

Wave moments for a signal sequence and its corresponding cepstral sequence are related in a way 
which obviates the need for direct calculation of cepstral coefficients. Hence an ideal calculation for 
moment of cepstrum is possible even if the duration of cepstrum is infinite. Calculation oi cepstral 
coefficients as such requires sophisticated phase unwrapping algorithms and is essentially prone to 
problems of aliasing. It is also possible to describe at least theoretically all the properties of a signal 
sequence from its wave moment. As an application of this, wave moments of mimmum phase sequences 
are calculated from the corresponding linear phase sequences without actually calculating the minimum 
phase sequence. 


On Model-Fitting for Fast-Sampled Data 

Rajiv Vijayan and H. Vincent Poor Department of Electrical and Computer Engineering and the 
Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, 1101 W Springfield Ave, 
Urbana.IL 61801 

The conventional discrete-time autoregressive model is poorly suited for modeling senes obtained 
by rapidly sampling continuous-time processes. The extreme ill-conditioning of the covariance matrix to 
be inverted in such cases causes numencal instabilities in the Levinson algorithm for estimating the 
autoregressive parameters. An alternative model, based on an incremental difference operator rather than 
the conventional shift operator, has been developed recently by the authors jointly with Goodwin and 
Moore. As the sampling interval goes to zero, the parameters of this model converge to certain parameters 
which depend on the statistics of the continuous-time process. A Levinson-type algorithm can be 
employed for efficiently estimating the parameters of this model. In this paper, the properties of this and 
related difference-based algorithms are explored both analyucally and numerically. 
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SESSION TP4 


QUANTIZATION D 


Asymptotics of Quantizers Revisited 

Ldszld Gyorfi, Tam is Linder, and Edward C. van der Meulen Hungarian Academy of Sciences, 
Technical University of Budapest, H-llll Budapest, Stoczek u2, Hungary, and Department of 
Mathematics, Kathoheke Universiteit Leuven, Celestijnenlaan 200 B, B-3030 Leuven, Belgium 

In this paper we reanalyze the results by Gish and Pierce (1968) on asymptotically efficient 
quantizing. In particular, asymptotic bounds are derived on the difference between the entropy of the 
uniform quantizer and that of the optimal quantizer when the mean square error becomes small Hereby, 
no assumptions are made at all on the density of the random variable being quantized, and use is made of 
some classical results by Rdnyi (1959) and Csisir (1973). Also, following the work by Ziv (1985), non- 
asymptotic and distribution-free bounds on the difference between the entropy of the uniform quantizer 
and that of the optimal quantizer are denved if both have the same distortion Finally,, non-uniform 
quantizers are considered. For the latter case the asymptotic relation is investigated between the entropy 
of the quantizer and the entropy of the random variable being quantized, with no assumption at all on the 
density. 


Low Dimension/Moderate Bit Rate Vector Quantizers for the Laplace Source 
Peter F Swaszck Department of Electrical Engineering, University of Rhode Island, Kingston, Rl 02881 
In this paper we present an unrestricted vector quantizer for the independent Laplace source 
x(x e R *), a source model useful for speech and image coding The VQ is based upon Hclmcrt’s 
transformation 


* = -j=i 1*1 

w t-1 


1 

yio+T) 


£ u.i-jivti 


1 = 1,2, ,k~ 1 


and consists of a scalar quanuzer for g and a lattice-based uniform VQ on a dimension k -1 simplex, 
a k-i(g), for the u,. The VQ is unrcstneted in that the resolution of the lattice VQ for the u, vanes with 
the result of the g quantization. 

Asymptotic performance and design results are reviewed Fischer’s pyramid vector quantizer is 
shown to be a special case of our VQ; low-dimension results for the PVQ arc provided. Implementation 
details of the VQ using vanous lattices are presented. Due to the geometric complexity of an exact 
analysis, an approximate design algomhm for finite bilratc is developed Example designs with Monte 
Carlo performance simulations are presented for low dimension and moderate bitrates (2 through 4 bits 
per dimension) to demonstrate the utility of this approach. 


Source/Channel Coding for Vector Quantizers by Index Assignment Permutations 

Kenneth Zeger, Erdal Paksoy, and Allen Gcrsho Dept, of Electrical and Computer Engineering, 

University of California, Santa Barbara, CA 93106 

An index assignment function for a vector quantizer in the presence of channel noise is a 
permutation of the quantizer codcvector indices. Each output index from a vector quantizer is mapped by 
an index assignment function to a new index which is subsequently used as the input to a block channel 
coder. For a given vector quanuzer (source coder) and a given error control block channel code, the 
choice of an index assignment function can have a profound effect on the average distortion achieved. 

This paper shows that existing procedures for the design of index assignment functions in the zero 
redundancy case can be generalized to include the use of block channel coding of the permuted indices. 
An effective design method is introduced for constructing index assignment functions for arbitrary block 
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error control codes with the objective of minimizing average distortion. Extensive numencal results for 
first-order Gauss-Markov and speech sources and for systematic linear codes with varying code rates 
demonstrate substantial performance gain at various channel error rates. Often, a gap of several dB of 
signal-to-noise ratio can exist between a poor and a good index assignment function. One interesting 
consequence is that a good index assignment for a quantizer with no channel with no coder can often be 
superior in performance to a poor index assignment for the same quantizer using redundancy coding. 


Optimal Quantization over a Finite-State Noisy Channel 

Hong Shen Wang and Nader Moayeri Wireless Information Networks Laboratory, Department of 
Electrical and Computer Engineering, Rutgers University, Piscataway, NJ 08855-0909 

The problem of quantizer design when the quantizer output is transmitted over a finite-state noisy 
channel is considered. Such a channel is a good model for fading channels, and the channel state 
information may be available to the receiver through signal-to-noise ratio measurements. Necessary 
optimality conditions are derived for the encoder when the decoder is fixed and given, and vice versa. In 
the absence of channel state infoimation at the decoder, the channel is simply equivalent to a discrete 
memoryless channel, and previous results and methods are applicable. When side mformafion is 
available, it can be used by the decoder along with channel outputs to better reproduce the source 
informatioa In either case, the optimality conditions lead to an iterative design algorithm. Numerical 
results are presented showing the performance of the system for various sources, channel models, and 
transmission rates, ft is shown that side information may lead to noticeable improvements in 
performance. 


Reduced Complexity Entropy-Pruned Tree-Structured Vector Quantization 
Tom Lookabaugh, Compression Labs, Inc, I860 Junction Avenue, San Jose, CA 95134 

Vector quantization is a data compression technique in which a vector of source symbols is 
represented by a reproduction vector drawn from a finite codcbook of such vectors The index of the 
chosen vector is all that need be communicated. Since the probability distribution across codcbook 
indices is not uniform, additional compression can be achieved by application of vanable rate noiseless 
entropy coding to the indices. 

Performance gains can be achieved if the vector quantizer and entropy code are designed jointly 
instead of sequentially. Entropy-pruned tree-structured vector quantization is a computationally attractive 
jointly designed system in which a large tree-structured vector quantizer is pruned so as to produce the 
smallest distortion among ail pruned trees of the same or lesser entropy In this paper, I show how binary 
arithmetic coding applied at each node of the tree search produces a system that is capable of progressive 
transmission and is also well suited to buffer-instrumented communicauon systems In fact, the 
multiplication free Q-coder arithmetic code combined with a piecewise linear distortion function in the 
vector quantizer leads to a multiplication free encoder with good rate-distortion performance and 
significant flexibility. Simulations are performed for image coding experiments. 


Optimal Scalar and Vector Predictive Quantizer Design 

Amitava Ghosh and fames George Dunham Department of Electrical Engineering, Southern Methodist 
University, Dallas, TX 75275 

Differential Pulse Code Modulation (DPCM) systems are important systems for data compression 
of speech, images and other signals. An optimal DPCM system based on optimization theory and Lloyd- 
Max quantizers is developed. Its performance is compared to a matched DPCM system for first order 
Gauss-Markov sources with a mean square error performance criterion. The performance of the optimal 
DPCM system is found to be closer to the distortion rate bound at high bit rates than the conventional 
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DPCM. The theory is extended to the design of infinite-state vector predictive quantizers. Unlike die 
conventional memoryless and finite-state vector quantizers, whose design is based on long training 
sequence, a probability density function is used when designing infinite-state vector predictive quantizers. 


The Asymptotic Distribution of the Error in Scalar and Vector Quantizers 

Don H. Lee and David L. Neuhoff Electrical Engineering and Computer Science, The University of 

Michigan, Ann Arbor, MI 48109 

An approximate formula is derived for the probability density of the quantization error caused by 
scalar and vector quantizers with many quantization levels. The rth moment of this density yields 
Bennett’s integral and its generalizations. Like Bennett’s integral, the formula becomes asymptotically 
exact as the number of levels increases. An application to two-stage quantization is discussed. 


Vector Quantizers Using Permutation Codes as Codewords 

Luzheng Lu, G. Cohen, and Ph. Godlewski Dept of Mathematics, University of Toulouse, Le Mtrail, 5, 
alle'es Antonio-Machado, 31058 Toulouse Cedex, France, and ENST, 46, Rue Barrault, 75013 Paris, 
France 

A new vector quantizer structure is proposed for reducing the coding complexity, memory 
requirements and the design computational burden, the Codcbook of such a quantizer consists of the first 
words of M(M £ 64) permutation codes. The encoding operation is then done by the minimum distortion 
word search and the fast permutation coding, the quantizer is uniquely defined by jointly specifying the 
first word set and the corresponding partition of the input space, so it can be constructed from an initial 
fixed first wond set by a LBG Algorithm and by the simple permutation code design techniques, the 
simulation results for such new quantizers designed and tested on different signals are given. 
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SESSION TPS 


CYCLIC CODES 


New Bounds on Cyclic Codes from Algebraic Curves 

J. Wol&nann G£.C.T , Universite'de Toulon, S31 JO, La Garde, France (40 min.) 

Starting from a deep link between the words of cyclic codes and plane algebraic curves over finite 
fields we use bounds on the number of rational points of these curves to obtain general bounds for the 
weights of cyclic codes. 


Algebraic Decoding Beyond epcH of Some Binary Cyclic Codes 

Jeanette Janssen Facultad de Ingemerid, Prolongacio’n Tampico slit, Adpo Postal 215, Sue "A", 
Salamanca, Gto 36730, Mexico 

It is well known that many cyclic codes have a true error-correcting capability e that is stnctly 
greater than the error-correcting capability esc;/, that follows from the BCH-bound. The well-known 
Berickamp-Massey decoding algonthm decodes only up to esc;; errors. A generalization of the 
Berlckamp-Massey algonthm exists that decodes in certain cases up to en M „ a lower bound on e coming 
from the Roos bound. M. Elia desenbed an algebraic decoding algorithm for the binary Golay code that 
uses the full error-correcting capacity of this code. In this presentation similar, but sometimes more 
complex, algebraic decoding algorithms, that decode up to e' errors, esc;; < e' < e, arc presented for a 
number of other binary cyclic codes. One of these algorithms is derived in detail, to show the way in 
which these kinds of decoding schemes can be found. 


Decoding Binary 2D Cyclic Codes by the 2D Berlekamp-Massey Algorithm 

Shojino Sakata Department of Production Systems Engineering, Toyohashi University of Technology, 

Tcmpaku, Toyohashi 440, Japan 

Binary 2D cyclic codes are a two-dimensional generalization of binary cyclic codes While binary 
cyclic codes arc constructed in terms of the univariate polynomial ring JC[xl over the binary field 
K :=GF(2), binary 2D cyclic codes arc in terms of the bivanate polynomial nng K[x,y], In this paper, 
we present a method of decoding binary 2D cyclic codes by using the 2D Bcrlckamp-Masscy algorithm 
which has been introduced as an extension of the Bcriekamp-Masscy algonthm to two dimensions, and 
discuss the error correcting performance of some 2D cyclic codes. As is the case with Stevens’ approach 
of extending the BCH decoding procedure to cyclic codes more general than BCH codes, we need some 
trial and error, i.e„ testing a certain number p of field elements of an extension field K ,= GF( 2 W ). We 
verify some merit of our approach by showing several simple examples of 2D cyclic codes Some arc not 
equivalent to any (ID) cyclic codes, and the other, which are equivalent to (ID) cyclic codes, have 
smaller values of p than when they are decoded by the ID Stevens method. 


On Error and Erasure Decoding of Cyclic Codes 

H. Shahri and K. K. Tzeng Department of Electrical Engineering, Lafayette College, Easton, PA 1S042, 
and Department of Computer Science and Electrical Engineering, Lehigh University, Bethlehem, PA 
18015 

This paper presents procedures for error and erasure decoding of cyclic codes up to the Hartmann- 
Tzeng bound as well as to special cases of the Roos bound. The main part is on converting the problem of 
error and erasure decoding to an error-only decoding problem so that the Fcng-Tzeng multisequence 
shift-register synthesis algorithms can be applied. This work has thus extended the result obtained by 
Stevens on error and erasure decoding of cyclic codes up to a special case of the HT bound. 
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Decoding of Cyclic Codes Beyond Minimum Distance Bounds Using Nonrecurrent Syndrome 
Dependence Relations 

G. L. Feng and K. K. Tzeng Department of Computer Science and Electrical Engineering, Lehigh 
University , Bethlehem, PA 18015 

The decoding capabilities of algebraic algorithms such as the Berlekamp-Massey algorithm, the 
Euclidean algorithm and out generalizations of these algorithms are basically constrained by the 
minimum distance bounds of the codes. Thus, when the actual minimum distance of the codes is greater 
than that given by the bounds, these algorithms cannot fully utilize the error-correcting capability of the 
codes. The limitation is seen to be rooted in the original Peterson decoding procedure followed by these 
algorithms which requires the determination of the error-locator polynomial from the Newton’s idennties 
which in turn require that the syndromes be contiguous resulting in a set or sets of linear recurrences. In 
this paper, we introduce a procedure which breaks away from this restriction and can determine the error 
locations from nonrecurrent syndrome dependence relations. This procedure employs a fundamental 
iterative algorithm, which we have introduced to denve the Berlekamp-Massey algorithm and its 
generalization, and an error-evaluation formula which is a generalization of Forney’s. It can decode many 
cyclic codes up to their actual minimum distance an is seen to be a generalization of Peterson's 
procedure. 


A Transform Based Decoding Algorithm for Cyclic Codes Via Non Preserving Permutations 
R. M. Campello de Souza Dept, of Electronics and Systems Commumcalions Research, Group - 
CODEC, Federal University of Pernambuco, Cidade Universitaria, 50741, Recife-PE, Brasil 

A new transform domain based decoding algorithm for cyclic codes is introduced. The technique is 
based on shortened syndrome look-up tables and makes use of preserving as well as of non-preserving 
permutations. The results presented in the paper are an extension of some previous work done on the 
subject and although only binary codes arc considered, the method can be generalized in a straightforward 
manner to nonbinaiy codes. 


Metacyclic Cooes 

Roberta Evans Sabin Computer Science Department, Loyola College, Baltimore, Maryland, and 
Computer Science Department, University of Maryland, Baltimore County, Catonsville, Maryland 

Since many of the most familiar linear error-correcting codes are ideals in group algebras, we wish 
to examine the structure of codes in group algebras based on non-abelian metacyclic groups. A code in 
such a binary group ring is either a two-sided or a one-sided ideal When the group is of odd order, the 
ring is semi-simple and decomposes into a direct sum of minimal codes. Unlike the abelian case, 
however, the decomposition is not unique. Minimal components arc isomorphic but are not necessarily 
combinatorially equivalent as codes. Like abelian codes, these codes have a simple algebraic structure 
which will aid in implementation. In most cases, these codes are found to be a significant improvement 
over abelian codes of the same length and dimension. 


82 



SESSION TP6 


CODING THEORY IV 


Linear Inequalities for Covering Codes 

Zhen Zhang Communication Sciences Institute, Electrical Engineering-Systems, University of Southern 
California, Los Angeles, California 90089-0272 

This paper deals with K ( n,R ), the minimal number of codewords in any binary code of length n and 
with covering radius R This quantity has been previously studied by Cohen et al and van Wee. We 
improve the lower bounds on K ( n,R ) for over 100 pairs of n and R within the range n < 33 and R < 10 by 
proving linear inequalities satisfied by covering codes of the following form 

fc(EAA)*pi, 

where lc is the indicator of the code C, I is the 2"-vector (1,1, , 1) and B, is a 2" x2" 0-1 matrix defined 
as follows 

B, = (A.ylje f;>€ Fi . 

where b ly = 1 \lfd(x,y) = i and 0 otherwise Some improvements for the lower bounds on t\n,k] are also 
obtained as by-products. 


Covering Radius Problems and Character Sums 

A Tictava'incn Department of Mathematics, University of Turku, SF-20500 Turku. Finland 

Using the Carlitz-Uchiyama bound for character sums Anderson found a lower bound for the 
minimum distance of the dual of a BCH code In this paper an upper bound is given for the covenng 
radius of that code and in fact of any code with large dual distance. 


Lower Bounds for Binary Covering Codes 

hro Honkala Department of Mathematics, University of Turku, SF-20500 Turku 50, Finland 

We give some modificauons of the van Wee lower bounds on K ( n.R ), the minimum cardinality of a 
binary code of length n and covenng radius R We use results about the classical combinatonal problem 
of covenng pairs by i-tuples. Our results give many improvements to the best previously known lower 
bounds on K(n,R). We also study j-subjectivity and covenng radius, and normal and subnormal codes 


On the Covering Radii of Codes over GF(q) 

H. Janwa Department of Mathematics, Caltech, PaS'tdena, CA 91125 

Let R denote the covenng radius of a q-sry [n.k.d] code We give an upper bound on R (which 
resembles the Griesmer on n), with many examples of codes for which tins bound is exact. For a class of 
codes, e.g, the optimal codes, we give an improvement of this bound. We give necessary and sufficient 
conditions for optimal codes to attain this bound. We present a lower bound on the algebraic geometric' 
codes in terms of the rational points and the genus of the underlying curve and show that for many of 
these codes the lower bound coincides with our upper bound. These codes furnish many non-uivial 
examples of ?-ary normal codes. We also give a critenon for a code to be abnormal. As an application, 
we give further examples of non-binary abnormal codec. If the code is cyclic with the generator 
polynomial g(x), then we give an upper bound on R involving g(x) Furthermore if g(x) is irreducible, 
then upper bounds on R are given using Wanng’s problem in finite fields We find exact values of 
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covering radius and minimum distance of some codes of this type including some 1-error-correcting 
quasi-perfect codes 

We give a generalization of the so-called "Norse bounds" for binary codes to <?-ary codes. We 
conclude with a discussion of further work and a list of open problems 

Some Results on the Covering Radius of Codes 

Xiang-dong Hou Department of Mathematics, University of Illinois at Chicago, Chicago, IL 60680 

Let t[n,k) be the smallest covering radius of any [n.it] binary code. We determine the values or 
improve the lower bounds of several entries in the table of t [n,k ] (n 2 64) by Graham and Sloane 

([19,9] = 4, ([38,6] 2 13, t [39,15] 2 8, r[40,6] 2 14, r[42,15] > 9, 
r[43,6] 2 15, r[43,26] = 5, r[45,6] 2 16,+r[47,7] 2 16, r[51,33] = 5, 
r[52,7] 2 18, t[52,18] >11, ([52,34] = 5, ([57,7] 220, ([58,20] 2 12, 

([59,8] 2 20, ([59,44] = 4, ([62,36] =7. (164,8)2 22 

These are done through two inequalities which improve a result of van Wee 

We also show that for any binary linear code C with covering radius 2 3, its norm N(C) introduced 
by Graham and Sloane satisfies 

N(C)<3tf-2. 

This improves the upper bound of Adams by 3, and implies that C is normal if R = 3 (Normality means 
W(C)S2R + 1.) 


Joint Decoding and Phase Estimation Via the Expectation-Maximization Algorithm 

Ghassan Kawas Kalch Ecole Nationale Superieure des Telecommunications. Departement 

Communications, 46, rue Barrault, 75634 Pans Cedex 13, France 

The Trellis-Coded Modulations arc very sensitive to carrier phase offset. To get a significant part of 
the predicted coding gain, a reliable earner reference is required. We present an iterative method for joint 
phase estimation and symbol decoding via the Expectation- Maximization (EM) algorithm Carrier phase 
offset and noise vanancc estimauon arc based on the maximum likelihood entenon. Esumatcs arc 
obtained via an iterative maximization of the Kullback-Lciblcr information measure. The Markovian 
properties of the encoder states sequences are used to calculate the required likelihoods At the end of 
iterations, the likelihood functions calculated by the EM algonthm can easily give optimum decisions an 
information symbols which minimize the symbol error probability The method can be applied to all 
codes that can be represented by a trellis Simulauon results will be presented 


Generalized Identity-Guards Algorithm for Minimum Distance Decoding of Group Codes in Metric 
Spaces 

L. B. Levitin, M. Naidjate, and C. R. P. Hartmann Boston University, College of Engineering, 44 
Cummington Street, Boston, MA 02215 

A generalization of the zero-guard algonthm for any group codes in metne spaces with a group 
invariant metric is suggested. The algorithm makes use of a special subset G 0 of codewords, called 
identity-guards. Only these codewords (which can be precomputed) should be stored and used in the 
decoding procedure. In general, the number of idcntiiy-guards is a small fraction of the number of all 
codewords. Necessary and sufficient conditions for a set of codewords to be G 0 arc found for the general 
case and, more specifically, for special cases, e.g. Strictly discrete spaces, soft decision decoding, 
arithmetic an codes, q-ary linear codes, permutation codes. 
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On the Linear M-Algorithm 

Harro Osthoff, Rolf Johannesson, Ben Smeets, and Han Vinck Department of Information Theory, 
University of Lund, Box US, S-221 00 Lund, Sweden, and Eindhoven University of Technology, P 0. Box 
513,5600 MB Eindhoven, The Netherlands 

The linear complexity version of the Af-algorithm (called LAf-algorithm) which we presented at the 
1SIT88 in Kobe is investigated further. For binary rate R = 1 12 convolutional codes our algorithm extends 
m each decoding step M possible information sequence estimates. The M most likely of 2M estimates are 
found by searching for the median. The storage size needed to decode an information frame of length / is 
2 Ml bits and it is organized in such a way that our final estimate can be found by a simple trace back 
method. The main problem with the LAf-algorithm is the correct path loss 

We have investigated different methods to recover the lost correct path Several quick-look-in 
codes, non-systemauc codes in feedback realization, and systematic codes with good profile spectrum 
were tested. These systematic codes show superior spontaneous path recovery 

We will also report on simulations of the LM-algonthm when it is used together with a systematic 
code for communication over an 8-lcvcl quantized AWGN channel When we suspect a correct path loss 
the decoder is reset with the M most likely states 
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SESSION TP7 


ERROR-CORRECTING CODES I 


Bounds on the Undetected Error Probabilities of Linear Codes for Both Error Correction and 
Detection 

Mao-Chao Lin Dept, of Electrical Engineering, National Taiwan University, Taipei 10764, ROC. 

In this paper, we study the (n, k, d 2 3) binary codes, which are used for correcting every single 
error and detecting other error patterns over the binary symmetric channel. We show there exists one code 
such that its probability of undetected errors is upper bounded by n + 1)[2 < "'* ) -n]' 1 We also study 
codes of length n for correcting all the error patterns of weight at most Xn and detecung other error 
patterns We show that there exists an («, Rn, dZ2Xn + 1) binary linear code whose probability of 
undetected errors is upper bounded by as n approaches infinity, where //(•) is the entropy 

function and 1 -R > H(2X). 


A New Class of Random Error Correcting Codes 

Sandip Kundu Thomas J Watson Research Center, PO Box 218, Yorktown Heights, New York 10598 
This paper considers the design of binary block codes that arc capable of corrccung up to 2 
symmetnc errors. These codes are based on similar equations as BCH codes, but their efficiencies differ 
Efficiencies vary from matching the best known codes to being inferior to BCH codes The construction 
pnnciple is novel and the author contends that decoding is simpler 


Asymmetric Error Correcting Codes 

Bella Bose Department of Computer Science. Oregon State University. Corvallis, OR 97331-3902 

Non-linear but cyclic codes capable of corrccung asymmetne errors arc desenbed For these codes 
the syndromes directly give the symmetnc functions of the error locations and so these codes arc much 
easier to decode. The hardware implementation of the decoding aigonthm is given In many cases the 
information rate of these codes is as good as or better than the corresponding BCH codes 


On Codes Correcting/Detectirg Symmetric, Unidirectional, and/or Asymmetric Errors 
J H. Weber, C. dc Vrocdt, and D E Bochcc Delft University of Technology, Department of Electrical 
Engineering, and Department of Mathematics anil Informatics, 2600 Delft, The Netherlands 

A code is called r r SyEC r 2 -UEC r 3 -AsEC d r SyED d 2 -UED d 3 -A.sED (with t ,<r 2 <r 3 , 
d\ Sd 2 £if 3 , 0£r, £d,) if it can correct up to t\ symmetnc errors, up to r 2 unidirectional errors, and up to 
t) asymmetne errors, as well as delect from r, +1 up to if, symmetnc errors that are not of the 
unidirectional type, from r 2 + 1 up to d 2 umdirccuonal errors that are not of the asymmetne type, and 
from r 3 + 1 up to dj asymmetne errors 

In this paper we denve necessary and sufficient condemns for a code to be (pSyEC r 2 -UEC r 3 - 
AsEC d i-SyED d 2 -UED d 3 -AsED Hence wc can obtain necessary and sufficient conditions for 
correction and/or detection of any combination of symmetnc/umdircctional/asymmetnc errors by making 
appropriate choices for r, and d,. Tnis includes existing as well as new results that appear as special cases 
of this general result For example, the conditions on a code to be /-SyECJ-UED (with 0<r£d) can be 
obtained by subsututing t\ = r 2 = rj=</, = r and d 2 = d 3 = J Further, it follows from the genera! 
necessary and sufficient conditions that some codes have stronger error correcting/detccting capabilities 
than they were originally designed for 
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Theory and Construction of M-ary Error Correcting and Discriminating Codes 
Kohichi Sakaniwa, Tae Nam Ahn, and T. R. N Rao Department of Electrical & Electronic Engineering, 
Tokyo Institute of Technology, Tokyo, and The Center for Advanced Computer Studies, University of 
Southwestern Louisiana, Lafayette, LA 70504-4330 

Hamming and Lee distance have been very well established and unlized to define error sets and to 
construct error control codes. However, as Berlekamp has correctly pointed out (in his book Algebraic 
Coding Theory ), neither of these metrics fit the mulu-level communication system using amplitude 
modulation under Additive Gaussian Noise (AGN) assumption. 

Therefore, we first derive some generalized metrics (called quasi-metrics ) defined for the n-tuples 
over the integer nng that are applicable to many actual channel models including multi-level amplitude 
modulation schemes with AGN We then develop a new general class of error control codes, namely 
error correcting and discriminating codes The error correcting and/or detecting codes become special 
cases of this general class of codes. In this context we establish another new concept called error 
difference set, £*, and relate its maximum quasi weight Q m „(£’) to the error correcting and 
discriminaung capabilmcs of codes 

Finally, we give m-ary code construcuon techniques and establish the error control capabilities 
using the theory developed. In a second construction method, we show how the error disenminating 
property can be used to construct unidirectional error control codes using asymmetne codes 


A Construction Method for Multilevel Error-Correcting Codes Based on Absolute Summation 
Weight 

Hajimc Jinushi and Kohichi Sakaniwa Department of Electrical and Electronic Engineering, Tokyo 
Institute of Technology 

By introducing the generalized Hadamard matrix (GHM), we present a new construcuon method 
for multilevel error correcting codes based on absolute summation weight the proposed code has two 
special features: 1. Decoding is very simple, i c„ errors can be removed by first multiplying the received 
word by the inverse GHM arid then rounding off the every digit of the transformed word 2. Since we can 
get a variety of GHM’s, multilevel error-correcting codes with vanous t (error-correcting capability) arc 
easily obtained. Moreover, we show by computer simulation that a much better performance can be 
obtained by the transmission system employing the proposed code compared to the uncodcd binary one 


On the General Error-Correcting Capability of Linear Codes 

Hans-Andrca Lochgcr Institute for Signal and Information Processing, £77/ Zcntrum, CH-8092 Zurich, 
Switzerland 

Linear (n,k) codes over GF(q) arc considered for use m correcting the error patterns e(e e GF (q) n ) 
in a specified set £ of addiuve error patterns A sufficient condition for the existence of a linear code that 
corrects all error patterns in £ is denved, this condition depends only on the cardinality |£ | of £ and 
generalizes the usual Gilbert-Varshamov bound for linear codes By a simple union bound, it is funher 
shown that, for arbitrary £ and arbitrary probability distnbution on the full set of error patterns, the 
average block error p robability, conditioned on the event that the error pattern is in £, over all linear ( n,k) 
codes satisfies P t lE < 'A | £ | By letting £ be a set of typical error patterns, this result can be used to 
show that linear codes can achieve capacity on a broad class of additive channels 


Cluster-Error-Correcting Array Codes 

P. G Farrell Electrical Engineering Department, University of Manchester, MI3 9PL, UK 

Array error control codes are linear block and convolutional codes which arc constructed from 
several single parity check or other component codes, assembled in two or more dimensions or directions, 
with emphasis on simple component codes and low complexity methods of decoding. Array codes are 
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particularly suitable for detecting and correcting two-dimensional bursts or cluster of errors. Clusters, 
patches or two-dimensional burst of errors occur in many digital transmission, processing or storage 
systems, wherever data is foimatted in two dimensions (e.g., magnetic tape, etc.). Efficient array codes 
can be devised for correcting cluster errors in such two-dimensional data structures 
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PLENARY SESSION 
Wednesday, 8 - 8:50 a.m. 


Routing in Interconnection Networks 

Bruce Hajek Coordinated Science Laboratory and the Department of Electrical and Computer 
Engineering, University of Illinois at Urbana-Champatgn, Urbana, IL 61801 

Packet-switched interconnection networks, such as the one connecting 65,000 processors in the 
Connection Machine, offer the possibility of programmable connections between processors Processors 
that are not directly connected communicate via multiple-hop routes, so that the processors appear to be 
connected at a higher logical level Special purpose hardware design and a variety of dynamic control 
devices are used to gain speed. A similar style of packet switching has also been proposed for wide area 
networks. Traffic congestion must be made to quickly diffuse through the use of fast, local mechanisms 
We will discuss mathematical and information-theoretic tools for analysis and modeling, with the goal of 
both understanding the general design problems and of suggesting new architectures r or its solution 

We will focus on deflection routing, originally termed "hot-potato" routing, which is a technique for 
maintaining bounded buffers in a packet-switched communication network If, due to congestion at a 
switch, not all packets can be sent out along shortest paths to their destinations, some packets arc sent out 
on other links The penalty is an increase in the distance traveled by packets, and the reward is die 
simplicity of switch design resulting from the absence of large buffers and routing tables Traditional 
store-and-forwaid networks use extensive computation at the nodes to determine packet routes in order to 
use transmission bandwidth spanngly. In contrast, deflection leads to simple switches by making liberal 
use of transmission bandwidth 

Both the worst case and average case performance of deflection routing will be discussed. 
Approximate analysis of the transient and steady state behavior of deflection routing in hypcrcubc and 
shuffle-like networks leads to a model in which the progress of a typical packet is desenbed by a random 
walk on the network graph The condition that packets reach their destination in spite of deflections is 
analogous to the condition of low-error probability for communication over a noisy channel Bounds 
based on information-theoretic considerations arc given on the average delay of a packc' 
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TECHNICAL SESSIONS 


Wednesday, 9 a.m. -12 m. 


SESSION WA1 

COMMUNICATION THEORY H 

Non-Linear Sequences with Controilable Correlation and Complexity Properties 
K. M. Ibrahim Department of Electrical Engineering, University of Baghdad, Iraq 

The construction of non-linear pseudorandom (PN) sequences with controllable complexity and 
correlation properties is presented in this paper. Such sequences are widely used for communication 
systems to get higher data privacy and immunity against interference. The correlation properties of these 
sequences provide a simple method for synchronization between the transmitter and the receiver The 
synchronization time using these sequences is evaluated and compared with the conventional scnal search 
techniques. 


Asymptotic Behaviour of MFSK in Noisy Phase Channels 

Yeheskel Dallal and Shlomo Shamai (Shitz) Department of Electrical Engineering, Technion - Israel 
Institute of Technology, Haifa 32000, Israel 

The limiting perfoimancc of quadraucally detected M -ary frequency shift keying (MFSK) in the 
presence of both additive white Gaussian noise (AWGN) and phase noise for Af->~> is addressed. A 
unifying analytical treatment is presented relying on power moments characterization of the preliltcred 
noisy phase signal. The moments are readily evaluated for a Brownian and a piecewise constant 
Markovian phase process. The fundamental impairments due to phase noise, random signal suppression, 
and crosstalk are addressed It is shown that the error probability approaches zero with M for all 
information rates up to C, the maximum achievable rate. The optimum prcfiltcnng bandwidth and the 
deterioration due to a finite frequency spacing are elaborated. For the case of the Brownian phase, 
encountered in coherent lightwave communication, the results indicate that near quantum limited 
performance is achievable as long as the average count of photons received dunng the coherence umc 
related to the nonzero linewidth is sufficiently large An uppcrbounding ereor rate exponent (reliability 
function) E(R) is obtained. This exponent is a non negative decreasing convex function for all rates 
0SR SC, equals zero at exactly R =C and is linear for low rates 0<R<R c where R c is the cnticai rate. 
The associated terms E (/?), R c , C and R 0 =E( 0), i.c, the cut-off rate are investigated in the phase noise 
limited regime. 


A Noncoherent CPM-Detector That Uses A Reduced Set of Basis Functions 

Torgny Andersson and Ante Svcnsson, Telecommunication Theory, University of Lund, Box IIS, S-22I 

00 Lund, Sweden, and Ericsson Radar Electronics AB, Aerospace Division, Avionics and Missile 

Electronics, Sweden 

Bandwidth and power efficient modulation schemes has, during some years, gained in interest due 
to the limited frequency spectrum. Continuous Phase Modulation (CPM) is a class of modulation 
schemes, with a constant envelope, which can be made both very narrowbanded and power efficient 
simultaneously. 

In some applications noncoherent detection is preferred. This detector does not have to know the 
phase of the carrier. The optimum noncoherent MLSE detector for CPM is, however, quite complex. In 
the detector the received signal is filtered in a filterbank. In this paper, the number of filters in the 
filterbank is reduced by using a limited signal space in the CPM-detector. This limited signal space is 
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found by a Gram-Schmidt procedure, where the less in energy in each filter in the filterbank is minimized. 
The error performance is calculated by means of a minimum equivalent Euclidean distance. This 
equivalent Euclidean distance is calculated in the signal space for the mismatched detector. 

It is found that the number of dimensions in the mismatched detector can be reduced quite a lot, 
with just a small reduction in the distance compared to the optimum detector. For a quaternary 2 RC- 
scheme with h = 1/3, the number of dimensions is reduced from 32 in the optimum detector to 6 in the 
mismatched detector with a loss of less than O.SdB in the error performance. 


A New Kalman Filtering Receiver over Fading Multipath Channels 

P. H. G. Coelho Rua Mar Bittencourt, 120A dll, Rtachitelo - 20951, Rio de Janeiro-RJ., Brasil 

This paper proposes a new Kalman receiver over fading multipath channels in presence of additive 
Gaussian noise using for channel model a rational transfer function. 

The receiver has less parameters to be adjusted so its computing load is less demanding than the 
standard Kalman receiver. 

The frequency selective fading causes a severe intersymbol interference which is harmful 
particularly for high capacity digital radio systems. The tracking properties of the Kalman filtering jointly 
with the use of a rational transfer function (with less parameters than the usual model) for the channel 
promote a fast convergence for the equalizer algorithm in adverse conditions such as those experienced in 
fading multipath channels. 

Several examples are presented by means of a simulation of a digital radio communication system 
using a 16-QAM modulation over a frequency selective fading channel modeled by a two ray fading 
channel model. 

The performance of the proposed equalizer, m terms of error rate, is compared with two transveisal 
filter receivers, one using the gradient algorithm and other using the least squares method. The results 
indicate that the suggested receiver has a better performance particularly over the transversal filter 
receiver using the gradient algorithm. 


Optimum Soft-Decision Demodulation for ISI Channels 

S. Raghavan and G. Kaplan Center for Magnetic Recording Research, University of California, San 
Diego; La Jolla, Calif. 92095, and Qualcomm, Inc , San Diego, Calif 92121 

Two different schemes of soft-decision demodulation for channels with finite intersymbol 
interferance (ISI) in the presence of additive white Gaussian noise are analyzed. The first approach 
employs ideal interleaving; the second involves maximum-likelihood decoding (MLD) for channels with 
deterministic finite memory. In both schemes the cut-off rate R o of the discrete channel created by the 
soft-decision demodulator is chosen as tL: design cmcrion. Expressions for the optimal thresholds of the 
quantizer associated with the demodulator arc denved. Results for the channel with ISI from one pulse on 
each side of the current pulse are presented, and the effects of ISI and the number of quantization levels 
are demonstrated for both schemes The gain of the MLD approach over the interleaving scheme is shown 
quantitauvely. Finally the (1-D) channel with soft-decision demodulation is analyzed. Closed-form 
solutions are derived and variation of R o with the number of quantization levels is presented. 


Signal Processing in Channels with Intersymboi Interference 

Daniel D. Klovsky Electrotechnical Institute of Telecommunications, Kuibyshev 445071, USSR 

The paper deals with the communication system with test pulse and prediction (STPP), suggested 
by the author, in which the transmission of code symbols is interrupted periodically by lest pulses with 
guard space, the duration of which is equal to T =QT (T - transmission period, Q = [z M !T] ■ relative 
memory of channel, t m - lime extent of channel memory). This system allows to realize adaptive quasi- 
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coherent symbol-by-symbol message deciding receiver at the time interval = (1+0) T (D - decision 
time delay) in the channel with varying (unknown) parameters and intersymbol interference. 

On the base of STPP suboptimal (when 0=0) symbol-by-symbol receiving algorithm using 
decision feedback was designed in the case of the additive Gaussian quasi-wight noise in the channel. 

This STPP and Viterbi processors tree-like diagrams, error probabilities and complications are 
compared. 

The possibility of combination of demodulation and conversion code decoding in the STPP 
procedure is analysed. Such a possibility is also discussed for the optimal symbol-by-symbol algorithm in 
ti e channel with intersymbol interference. 


Quantization Noise Spectra 

Robert M. Gray Information Systems Laboratory, Stanford University, Stanford, California, 94305 (40 
min.) 

Uniform quantizers play a fundamental role in digital communications systems and have been the 
subject of extensive study for many decades. The inherent nonlinearity of quantizers makes analysis 
difficult. It usually has been accomplished either by assuming the quantizer noise to be a signal- 
independent, uniform white random process or by replacing the quantizer by a deterministic linear device 
or by combining the two assumptions Such linearizing approximations simplify the analysis and permit 
the use of linear-systems techniques, but few results exist quanufying how grid such approximations are 
for specific systems. These complications are magnified when the quannzer is inside a feedback loop, as 
in the case of the Sigma- Delta and Delta modulators. 

Exact descriptions of the moments and spectra of quantizer noise have been developed recently for 
the special case of Sigma-Delta modulators. These results demonstrate that the white noise and 
linearization assumptions can be quite poor approximations in some systems. It turns out that many of the 
techniques used in the analysis were first applied to the analysis of quantizers by Clavier, Panter, and 
Grieg 0947) in pioneering, but often overlooked work that preceded Bennett's (1948) classic study study 
of quantization noise spectra. 

We take advantage of the benefit of hindsight to develop several of the'c results in a unified and 
simplified manner. Exact formulas for quantizer noise spectra are developed and applied to a variety of 
systems and inputs, including scalar quantization (PCM), dithered PCM, Sigma-Delta modulation, 
dithered Sigma-Delta modulation, second-order Sigma-Delta modulation, and two-stage Sigma-Delta 
modulation. 
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SESSION WA2 


MULTIPLE ACCESS H 


Throughput and Delay Performance of a Channel-Sensing Coded Band-Limited Spread-Spectrum 
Multiple-Access Scheme 

Samuel Resheff and Izhak Rubin Electrical Engineering Department, 67310 Boelter Hall, University of 
California, Los Angeles, CA 90024 

Recent results indicate that wide-band nets using spread spectrum signals and operating under a 
CSMA access protocol (CSMA/SS) can be efficient, provided the bandwidth is wide enough so that the 
signal set is almost orthogonal. Signals' orthogonality eliminates packet collisions, and the CSMA/SS 
feature of the implementation can thus provide for better coordination of station transmissions. In 
particular, when half-duplex transceivers are used, a ready user will defer transmission when the channel 
is sensed busy, thereby unlocking its own receiver for possible message reception. However, as the 
number of stations in the net increases, while the total available channel bandwidth remains fixed, perfect 
signal orthogonality no longer prevails. Packet collisions become then the dominant factor in determining 
the network delay-throughput performance. Under such conditions, channel sensing can be used to 
provide information to stations as to when the channel is overly utilized. Such side-information can be 
obtained by using specially encoded message headers or by using an ancillary broadcast channel which is 
used solely to inform the net stations about the channel's activity level. In addition, error-correction codes 
are used to provide for correction of errors that occur due to simultaneous transmissions of non- 
orthogonal signals. In this paper, we introduce and study the message delay and channel throughput 
behavior of such a CSMA/SS scheme. It is assumed that each net station can gain information as to 
whether the total number of on-going transmissions exceeds a given threshold (M), or not. This entails, 
for example, the availability of an ancillary low-rale out-of-band or in-band signaling channel. A 
transmitting station will abort its transmission upon the reception of a signaling; message indicating that 
the current number of transmissions exceeds the prescribed threshold. A random-access scheme is used to 
control the access of messages to the channel. Using our derived performance equations, we present 
numerical results illustrating the delay-throughput performance of such a CSMA/SS scheme. Key 
parameters involved in this performance analysis include: channel bandwidth, error-correction code 
capability, and propagating delay. Given an average message length, an activity threshold level M can be 
selected to yield the best delay-throughput performance charactenstics. 


Capacity Region of a Waveform Gaussian Multiple-Access Channel 

Chao-Ming Zeng, Ning He, and Federico Kuhlmann Umversidad Nacional Autonoma de Mexico, 
Facultad de Ingemeria, Division de Estudios de Posgrado, P 0 Box 70-256,04510 Mexico, D.F. 

A waveform (continuous-time) Gaussian multiple-access channel with" average-power' constrained 
inputs is considered. That is, the sum of two input signals x i (r) and Xi(t) is passed through a linear filter 
//(/). The input need to satisfy the average-power constraints y X} ( t)dt £ P„ i=l, 2, for large T. The 

capacity region of this channel is obtained by using the orthonormal expansions of waveforms approach 
and a result for a parallel, discrete-time, Gaussian multiple-access channel. Our result shows that for 
Gaussian multiple access channels, there also exists a so-cailcd " water-filling " interpretation for multi¬ 
user encoding, similarly as in the classic (single input) Gaussian channel case. Finally, an optimum 
assignment of the power spectra without lime-sharing is given to attain the capacity line. 
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On Growing Random Trees in a Random Environment with Applications to Multiaccess 
Algorithms 

Dan Kessler and Moshe Sidi Electrical Engineering Department, Technion - Israel Institute of 
Technology, Haifa 32000, Israel 

Binaiy trees that grow in a two-state Markovian environment are considered. For such trees we first 
obtain the condition on the environment process so that the tree does not grow indefinitely, and then we 
calculate the expected number of vertices in the tree. These two problems are addressed for two different 
ways of growing the binary tree, and they are compared. The two ways differ in the order by which the 
vertices of the growing tree reproduce. 

The above results are applied to multiaccess networks in which the shared channel is noisy. We 
assume a slotted-time collision-type channel, Poisson infinite-user model, and a binary feedback. Due to 
the noise in the shared channel the received signal may be detected as a collision even though no message 
or a single message is transmitted (referred to as error). A common assumption in all previous studies of 
multiaccess algorithms in channels with errors is that the channel is memoryless. The above results are 
applied to the analysis of the operation of the tree-CRA with memory. We obtain the condition on the 
channel parameters for stability and the throughput of the algorithm when this condition holds. 


Polling Systems with Routed Customers 

Moshe Sidi and Hanoch Levy Electrical Engineering Department, Technion •-• Israel Institute of 
Technology, Haifa 32000, Israel, and Computer Science Department, Tel-Aviv University, Tel-Aviv 
69978, Israel 

A queueing network that is served by a single server in a cyclic order is studied. Customers arrive at 
the queues from outside the network according to independent Poisson processes. Upon completion of his 
service, a customer may leave the network, be routed to another queue in the network or rejoin the same 
queue for another portion of service. The single server moves along the different queues of the network in 
a cyclic manner. Whenever the server arrives at a queue (polls the queue), he serves the waiting 
customers in that queue according to some service discipline. Both the gated and the exhaustive service 
disciplines are considered. The service time of a customer has a general distribution (may be different 
from queue to queue). When moving from one queue to the next queue, the server incurs a switch-over 
period with a general distribution. 

For this general queueing network we derive the expected number of customers present in the 
network queues at arbitrary epochs, and compute the expected delays observed by the customers. In 
addition, we introduce a pseudo conservation law for this network of queues. Some interesting numerical 
examples illustrate how routes and server moving direction affect the performance of the network. 


On Gaussian Feedback Capacity 

Amir Dembo Information Systems Laboratory, Stanford University, Stanford, CA 94305 

Pinsker and Ebert proved that in channels with additive Gaussian noise, feedback at most doubles 
the capacity. Recently, Cover and Pombra proved that feedback at most adds half a bit per transmission. 
Upper bounds on the feedback capacity are obtained here using majorize lion tools. These bounds imply 
for noise covariance matrices with bounded away from zero and infinity eigenvalues that at high input 
SNR the difference between feedback and non-feedback capacities is marginal, while at low input SNR 
their ratio is close to one Specializing these results to stationary channels we recover some of the bounds 
recently obtained by Ozarow using different techniques. 
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Modified Viterbi Decoding for Convolutionally Encoded Hybrid-ARQ Protocols 

Stephen B. Wicker and Brace Harvey School of Electrical Engineering, Georgia Institute of Technology, 

Atlanta, GA 30332 

An error pattern in a convolutionally encoded data packet corresponds to a cycle containing the zero 
state in a weighted directed graph. The structure inherent in this error process can be used to define a 
type-I hybrid-ARQ protocol. Let the "transition trap length" J T be the minimum number of branches a 
path leaving the zero state must take before its weight is guaranteed to equal or exceed that of the 
minimum free Hamming distance dp,,. Note that the path need not terminate at the zero state. In the 
proposed modification to Viterbi decoding the rate of increase of the partial path metrics is monitored 
over a sliding window of Jj branches. A path is labeled unreliable if the rate of increase of its partial path 
metric exceeds a threshold r. If the maximum likelihood path is found to be unreliable, then a 
retransmission of the packet is requested: otherwise, decoding is allowed to proceed normally. Analysis 
and simulation results am presented to show that the proposed type-I hybrid-ARQ protocol substantially 
improves data reliability at the expense of a minimal reduction throughput. 


Coding Theory for Secret Sharing Communication Systems with Two Gaussian Wiretap Channels 
Hirosuke Yamamoto Department of Communications and Systems, University of Electro- 
Communications, 1-5-1 Chofugaoka, Chofu, Tokyo, Japan 

The coding theorem is proved for the secret sharing communication system (SSCS) with two 
Gaussian wiretap channels, which is an extension of both the SSCS with two noiseless channels and the 
Gaussian wiretap channel (GWQ system. The admissible region of rates and security levels for the SSCS 
with two GWCS’s is given by 

*1 ^ C Sl R\ J rCu l Ra 

•*2 S Cm,R\ + Cs 1 Ri 

H(S) S Cm.R. + CV.Rj, 

where Cm, and Cj ( (J = 1,2) are the channel capacity and secrecy capacity of GWC j, respectively, rate 
Rj is defined by channel-) symbols per source symbol, and security level is measured by the 
equivocation of wiretapper j. 


On the Characterization of Information Divergence 

G. Q. Shi Telecom Research Laboratories, 761-772 Blackburn Road, Clayton, Victoria 3168, Australia 
This paper presents some results on single letter charactenzation of information divergence 
For the discrete case, information divergence is degenerate as data are compressed, i.e., when one 
encodes the data in a composition class, the information, at the level of information divergence, is not, in 
fact, compressed. Therefore, by using a normal definition of coding rate, one cannot obtain 
characterization of the achievable single-letter solution for the information divergence. 

Here we first give a bent rate definition and then, by using it. we get some results for one variable 
and bivariables with data compression. For the problem of bivariables with two-side data compression, 
however, only bounds of the single letter characterization are obtained. 
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SESSION WA3 


SIGNAL PROCESSING D 


On the Estimation of the Order of a Stationary Ergodic Markov Source 
Chuangchun Liu Electrical Engineering Department, University of Maryland, College Park, MD 20742 
We study an estimator for the order of a stationary ergodic Maikov source. This estimator, which is 
easy to implement, is consistent and possesses exponential and near-exponential rates of decay of the 
probabilities of underestimation and overestimation respectively. 


Efficient Identification of Impulsive Channels 

Serena M. Zabin and H. Vincent Poor School of Electrical Engineering, Georgia Institute of Technology, 
Atlanta, GA 30332, and Coordinated Science Laboratory, University of Illinois, Urbana, IL 61801 

The man-made electromagnetic environment, and much of the natural one, is basically impulsive, 
i.e., it has a highly structured form characterized by significant probabilities of large interference levels. It 
has been demonstrated that the performance of communications, radar, and sonar systems operating in 
impulsive channels can be greatly enhanced if the statistics of the channel are known and exploited. 
Consequently, the problem of idenufying impulsive noise channels is a basic and important one. A 
physically-meamngful, parametric model for impulsive interference is the so-called Gass A Middleton 
model, whose parameters A and T can be adjusted to fit a great variety of non-Gaussian noise phenomena 
occurring in practice. The first parameter, A, referred to as the "Overlap Index," is a measure of the 
average overlap of successive emission events. The second parameter, F, the "Gaussian Factor,” is the 
ratio of the intensity of the independent Gaussian component of the input interference tc the intensity of 
the non-Gaussian component 

In this study, a batch estimator of the Gass A parameters with good small-sample-size performance 
is obtained. This estimator is based on the EM algorithm, a two-step iterative technique which is ideally 
suited for the Gass A estimation problem since the observations can be readily treated as "incomplete 
data.” For the single-parameter estimation problem (A unknown, T known), a closed-form expression for 
the estimator is obtained. Furthermore, for the single-parameter estimation problem, it is shown that the 
sequence of estimates obtained via the EM algorithm converges, and if the limit point to which the 
sequence converges is an intenor point of the parameter set of intercsi, then it must necessarily be a 
stationary point of the traditional likelihood function. The small-sample-size performance of the proposed 
EM estimator is also examined via an extensive simulation study. For both the single-parameter and two- 
parameter estimation problems, the results of this study indicate that this likelihood-based scheme yie'ds 
excellent estimates of the Gass A parameters (in terms of attaining the Cramdr-Rao Lower Bound) for 
small sample sizes (O(K) 2 )). 


Nonparametric Identification of a Cascade Nonlinear Time Series System 

Miroslaw Pawlak Department of Electrical Engineering, University of Manitoba, Winnipeg, Manitoba, 

Canada R3T2N2 

In this paper a nonlinear ume senes system of a cascade structure is identified. The system consists 
of a nonlinear mcmoryless element followed by a dynamic linear subsystem. Given a colored Gaussian 
input, the Hermite polynomials based method for the recovering of the system nonlinearity is presented 
The algorithm is carried out by solving an integral equation related to the identification problem. The 
consistency along with rates of convergence arc established. No assumptions concerning continuity of the 
characteristic or its functional form arc required. An extension to the case of an input Markov process 
possessing the bivanate density with a diagonal expansion is also made. 
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Application Criteria of the Pencil of Functions Method in ARMA System Identification 
Diamantino R. S. Freitas Faculdade de Engenharia da Universidade do Porto, D.EE.C., 4099 Porto 
Codex, Portugal 

ARMA system identification is generally considered to be a difficult nonlinear problem, the use of 
the Kalman equation error, as in the pencil of functions method (POFM), turns that into a linear 
alternative, yielding good results in low-noise situations, the identification of electroacoustic transfer 
functions, in high accuracy, low noise, conditions, is a domain well suited for the application of the 
POFM, given its robustness, consistence and low bias. However, the success in the application and the 
accuracy of the results depend, in a crucial way, on the appropriate selection of q, the pole of cascaded 
first-order low pass filters used in the method and n, the number of signal samples employed. This 
essential point has been, to our knowledge, paid little attention. The work presented shows that the error 
energy E presents a sharp minimum, this leads to the formulation of criteria: the optimum q value if close 
below the lowest of the system poles amplitudes; the optimum n value is a compromise between 
increasing errors in the estimated model numerator and decreasing errors in the denominator. 


Approximate Bayesian Classification Based Upon Hidden Markov Modeling 

Ncri Merhav and Yanv Ephraim Speech Research Department, AT&T Bell Laboratories, Murray Hill, 

NJ 07974 

We investigate a Bayesian approach to muluple hypothesis testing for hidden Maikov sources, 
whose statistics are given empirically by training data. The exact Bayesian optimal decision rule involves 
calculation of conditional means of probability density functions and hence it is computationally 
untractable. To avoid this difficulty, we propose an alternative decision rule which is computationally 
more attractive. It is proved that the asymptotic exponential rate of decay of the error probability, 
associated with the proposed decision rale, is opumal. Furtheimore, it does not require knowledge of the 
prior probability densities of the model parameter. The approach is generalized to hypotheses testing in a 
noisy environment, given training data of the clean sources and the noise process. Simulation results on 
computer generated hidden Markov processes reveal significant preference of the proposed approach over 
a standard method currently used. 


Imaging a Randomly Translating Object from Point Process Observations 

Donald L. Snyder and Timothy J. Schultz Electronic Systems and Signals Research Laboratory, 

Department of Electrical Engineering, Washington University. St. Louis, Ml 63130 

Measurements of a randomly translating object described by a spatial intensity function are modeled 
as a time-space doubly stochastic Poisson process in which the intensity function moves randomly in 
time. A method for producing constrained maximum-likelihood estimates of the object intensity is 
developed and examples for various characterizations of the mouon are presented. 


Robust Signal Reconstruction in a Hilbert Space Setting 

Richard 1. Barton Orincon Corporation, 9363 Townc Centre Drive, San Diego, CA 92121 

In this presentation, we examine the problem of reconstructing an unknown signal from noisy 
observations. We assume throughout that the signal is a member of a known reproducing kernel Hilbert 
space (RKHS) H so that the observation functionals L ,(f) = / (») are bounded. We assume also that the 
true values of the signal /o are known only to belong to some convex set C containing the observations 
(y(r), te 0 ). In order to generate a robust reconstruction, we seek a function fa H that solves the 
minwtax problem 

sug| | /-/||H = mf sug||/-g||Pi 
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where U = (feH ■:ll/ll h SW, {/(O, re 0 } e C ), and Af is an arbitrary constant We show that / is 
simply the minimum noim element of U closure, and we give some examples for which we car 
characterize/ explicitly. 


Asymptotics of Divergent LMS 

Todd F. Brennan Department of Electrical and Computer Engineering, University of Wisconsin, 1415 
Johnson Drive, Madison, WI53706 

Convergence properties of the LMS algorithm have been well studied [2,8,9,10], but few 
quantitative results are available for divergent LMS. Large deviation theory is used to compute several 
different LMS asymptotics. During the derivation, it is observed that LMS does not scale as a slow 
Markov walk [1], so a transformed process is used instead. Necessary and sufficient conditions for die 
existence of large deviation behavior in LMS are given. For certain inputs, it is proven that LMS diverges 
exponentially with probability one, and mean exit time can be computed in closed form. Convergence is 
shown using large deviation theory as well. Direct simulations confirm theoretical predictions. The 
method shown here is general, and can be used to handle the observation noise case and others. Most 
notably, these large deviation methods handle non-Gaussian inputs and Markov input dependencies, but 
may not admit closed form solutions for certain data dependencies. 


Double Sampling Af-Detection Procedure 

Liu Youhcng and Tang Chuanzhang Department of Radio Electronics, Peking University, Beijing, PJ{ 
of China 

A new detection procedure, the double sampling procedure, is developed in this paper. Applying it 
to the detection of signals in contaminated Gaussian noise, we obtain the double sampling M- detector. It 
exhibits simplicity in structure while retaining the robustness property of the Af-sequential detector. 
Simulation results are given that coincide with the analytical results quite well. 
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Entropy-Based Bounds on the Redundancy of Prefix Codes 

Padhraic Smyth Communication Systems Research, Jet Propulsion Laboratories 238-420, 4800 Oak 
Grove Drive, Pasadena, CA 91109 

We consider the problem of bounding the redundancy of binary prefix codes for discrete 
mcmoryless sources. Recent results have bounded the redundancy of Huffman codes in terms of the 
probability of various components of the source alphabet, e.g., the most likely letter. Similar results for 
alphabetic prefix codes have also been derived where the smallest probability component is known. We 
extend these results to the case where the entropy of the source can be estimated or bounded, and denve a 
variety of bounds for the redundancy of Huffman, Shannon-Fano, and weight-balanced (alphabetic) 
coding schemes, expressed in terms of source entropy. 


Efficient Representations for Huffman Coding 

Cheng-Chang Lu Department of Mathematical Sciences, Kent State University, Kent, OH 44242 

An efficient representation for the Huffman tree is proposed. The tree is uniquely specified by a 
sequence of decimal node numbers. The Huffman code can be determined from the node number easily 
by a shifting procedure, and the node number can also be generated from the corresponding code by 
adding a leading bit. A new data structure for static Huffman coding is developed using this 
representation. Based on the inherent parent-child property, an efficient dynamic Huffman coding 
algorithm can also be implemented without building a look-up table to keep track of all parents and 
children. 


A Universal Model Based on Minimax Average Divergence 

Cheng-Chang Lu and James George Dunham Department of Mathematical Sciences, Kent State 
University, Kent, OH 44242, and Department of Electrical Engineering, Southern Methodist University, 
Dallas, TX 75275 

For a non-adaptive data compression system, the source structure and statistics required by the 
coding unit have to be determined from the input sequence before the corresponding codes can be 
generated. This requires each individual input sequence to pass the source modeling part of the system 
first and then go through the coding part Such implementation may not be acceptable in terms of extra 
memory and ume required. In this paper, a universal model for a class of input sequences is proposed, 
which minimizes the maximum average divergence between the model and training samples. A 
theoretical searching algorithm for finding the minimax average divergence is developed, based on the 
property that the maximum average divergence can be decreased by modifying the model. Also proposed 
is a practical searching algorithm that can be easily implemented on digital computer to find the mir.imax 
universal model. 


A New Asymptotically Optimal Code of the Positive Integers 


Hirosuke Yamamoto and Hiroshi Ochi Department of Communications and Systems, University of 
Electro-Communications, 1-5-1 Chofugaoka, Chofu, Tokyo, Japan 


A new universal binary code of the positive integers is proposed as a modified version of Warg’s 
flag encoding scheme based on the bit stuffing technique The average codeword length defined by 
L(n)/2 M , where L(n) is the codeword length of n, is one bit shorter than Wang’s scheme if the 
flag’length /is two, and it is shorter than Wang’s if M £ 12 for/= 3, M £ 21 for/ = 4, and so on. The 
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performance of the new scheme is also compared with other universal schemes for geometrically o' 
Poisson distributed integers. The new scheme is almost as efficient as Capocelli's Fibonacci encoding 
scheme and is more efficient than other schemes. Moreover, the encoding and decoding algorithm is 
simpler than other universal schemes including Wang’s. Hence, the new flag scheme is suited for 
hardware implementation as well as software in many practical applications. Furthermore, an 
asymptotically optimal code can be realized by modifying the new flag scheme such that the flag length 
varies dynamically. 

Combined Equalization and Coding with Minimum Mean Square Vector Coding 
J. M. CiofS, J. S. Chow, and J. Tu Information Systems Laboratory, Stanford, CA 94305 

The combined design of equalization and coding methods is necessary to achieve the highest 
possible data transmission rates on channels with intersymbol interference. Existing approaches to 
combined equalization and coding are strictly zero-forcing in that intersymbol interference is eliminated 
through feedback in symbol-by-symbol approaches or through careful selection of successively 
transmitted blocks in block ("multitone" or "vector coding") approaches. In this paper, we investigate 
block approaches where some residual intersymbol interference is permitted in order to achieve an 
overall reduction in mean-square distortion (noise plus residual interference). 

The approach involves converting the ISI-channel into a canonical minimum-mcan-squared-error 
minimum-phase equivalent through the use of what is called a "mean-square whitened matched filter." 
this conversion leaves the overall system with the highest possible SNR. Further, the optimum applied 
transmit spectrum to such an equivalent channel is shown to be the "water-pouring" energy distributioa 
We then design a coset-coded multichannel modulation method with this spectrum for the equivalent 
mean-square-minimum-phase channel and apply it to the channel. We call this method "Minimum Mean 
Square Vector Coding," or MMSVC. MMSVC is shown to achieve transmission rates that are as close to 
capacity on the ISI-channel, as the equivalent coset code is to capacity of the "flat" (or "ideal") additive 
white Gaussian noise channel (2-4dB away from capacity), even at low to moderate SNR's where 
existing zero-forcing methods do r.ot apply as well. 


Bounds to the Capacity of Discrete Memoryiess Channels with Input Constraints 

Aii Khayrallah and David L. Ncuhoff Electrical Engineering and Computer Science, The University of 
Michigan, Ann Arbor, Ml 48109 

Bounds are presented to the capacity of a discrete memoryiess channel with input constraints. 
Specifically, Mrs. Gerber’s Lemma is used to derive a simple lower bound to the capacity of a binary 
symmetnc channel with the constraint that the infinite input sequence be a member of a specified 
subshift, this bound is a function only of the crossover probability of the channel and the capacity of the 
subshift. Secondly, by restricting the allowable input probability distributions (in the usual definition of 
nrt-order capacity), we obtain upper bounds to the capacity of a discrete memoryiess channel with inputs 
constrained by a subshift. One such rcstnetion is the requirement that the n-dimensional input distribution 
be stationary. Others derive from the maximum and minimum frequencies with which strings can occur 
in a member of the subshift. TIic bounds are evaluated for binary symmetnc channels and (d,k) run- 
length constraints. Compansons are made with the recent bounds of Zchavi and Wolf. 


The Adaptive Guazzo Algorithm 

Gtfrard Battail Ecole Nationale Supe'rieure des Telecommunications, De'partement Communications, 46 
Rue Barrault, 75634 Paris Cedex 13, France 

The Guazzo algorithm is a source coding algorithm which can easily be made adaptive in order to 
deal with unknown and/or varying source statistics. This paper desenbes the adaptive version of this 
algorithm, reports theoretical predictions of us performance and results of its simulation on both 
stationary and nonstationary sources, including actual computer files. 
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On the Optimal Inductive Inference Scheme from the View Point of Source Coding 
Toshiyasu Matsushima, Joe Suzuki, Hiroshige Inazumi, trod Shigeichi Hirasawa Dept, of Management 
Information, Yokohama College of Commerce, Yokohama 230, Japan; Dept of Industrial Engineering 
and Management, Waseda University, Tokyo 169, Japan; and Dept, of Information Science, Sagami 
Institute of Technology, Kanagawa 255, Japan 

In this paper, we discuss the inference of predicate logic, which is widely used for representation of 
knowledge in artificial intelligence (AI) systems, from the view points of source coding and decision 
theory. Since the inference in logic can be regarded as some kinds of information transformation, we can 
recognize an analogy between inference and source coding. Inductive inference is regarded as source 
, encoding, because observed facts or examples are compressed into an axiom similar to a source sequence 

into a codeword. On the contrary, deductive inference is interpreted as decoding. >From the view point of 
j decision theory, inductive inference is regarded as the decision problem selecting the collect axiom which 

represents an observing world. We propose a new inductive inference scheme which induces the mini- 
I mum Bayes risk. Moreover, we show the method for selecting the axiom which represent the finite 

, observed facts by the minimum description length code. 

I 


I 




SESSION WAS 


SHANNON THEORY HI 


Successive Refinement of Information 

William H. Equitz h formation Systems Laboratory of the Department of Electrical Engineering, 
Stanford University, Stanford, CA 94305; now at IBM Almaden Research Center, San Jose, CA 95120- 
6099 

We characterize problems in which optimal descriptions can be considered as refinements of one 
another. We do this because we may optimally describe a message with a particular amount of distortion 
and later decide that the message needs to be specified more accurately. If an addendum to the original 
message is then sent we hope that this refinement is as efficient as if the more strict requirements had 
been known at the start In general, we ask whether it is possible to interrupt a transmission at any time 
without loss of optimality. 

We present necessary and sufficient conditions for achieving optimal successive refinement and 
establish that all finite alphabet problems have the required properties for at least small distortions. 
Furthermore, we show that finite alphabet signals with Hamming distortion, Gaussian signals with 
squared error distortion, and Laplacian signals with absolute error distortion all satisfy these requirements 
over the enure range of possible distortions. On the other hand, we exhibit a family of simple 
counterexamples which show that successive refinement is not achievable in general. 


Maximum Entropy Charge Constrained Run Length Codes 

Kenneth J. Kerpez, Ayis Gallopoulos, and Chns Heegard Bell Communications Research, 445 South 
Street, Morristown, NJ 07960, Athens, Greece, and the Department of Electrical Engineering, Cornell 
University, Ithaca. NY 14853 

The maximum entropy distribution maximizes the code rate for a given channel constraint. Closed 
form expressions are known for the power spectrum of maximum entropy ran length limited (d,k) 
sequences and charge constrained (C) sequences. Informauon is coded into a sequence that satisfies both 
run length and charge constraints simultaneously; the charge constrained run length limited ( d,k,C) 
sequence. A DC null is implied by the charge constraint C, making it useful for a recording system with 
an AC coupled rotary head. An expressing for computing the maximum entropy distribution and its 
power spectrum is presented for a (c i,k,C) sequence. TTie expression involves the adjacency matrix of the 
variable length state transition diagram of the (< l.k.C) sequence. Simplified formulas arc given for the 
case of no AT constraint, and for the tight constraint k = d + 1. 


Gambling Using a Finite-State Machine 

Mcir Fcdcr 12 Shirat-Hatanur St, Henltya, 46420, Israel 

Sequential gambling schemes in which the amount wagered on the outcome of a random sequence 
is determined by a finite state (FS) machine are defined and analyzed. We assume that the FS machine 
determines the fraction of the capital wagered at each lime instance, i, on the outcome at the next time 
instance, l + 1, and that wagers are paid at even odds. We show that the maximal capital gam in any finite 
state sequential gambling scheme is given by, 


where So is the initial capital, S„ is the capital after time instance n, x is the outcome sequence and 
H fs (x), which is a measure based on the empirical entropy of x, is defined as the finite-state complexity 
of x.~ We also analyze a specific gambling scheme based on the Ziv-Lcmpcl method for universal 
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compression, and its performance provides a relation between the finite-state complexity defined above 
and the finite-state complexity defined by Ziv and Lempel. 


An Application of the Galileo Multidimensional Scaling System to Human Communication 
Walton B. Bishop University of Maryland, College Park, MD 20742 

The multidimensionality of effects that a message recipient’s prior knowledge has upon 
comprehension calls for new methods of analysis. Attempts to reduce multiple interpretations of a given 
message by using causal analysis proved to be unduly cumbersome. The Galileo multidimensional 
scaling system, however, provides a simple, yet precise, way of measuring some of the effects prior 
knowledge has on message interpretation. Data collected from a representative sample of message 
recipients, when analyzed by the Galileo system, will tell the message originator how to modify a 
message so that it will be interpreted correctly by its intended audience. The Galileo™ computer program 
has been used successfully in such diverse areas as mass communication, political communication, 
criminal justice, advertising, and marketing. It seems ideally suited for the unobtrusive measurement of 
the effects produced by a message recipient’s prior knowledge. It is these effects that usually interfere 
with the application of Shannon theory to human communication. (The research reported here was done 
as part of the author's doctoral dissertation under the direction of Professor Edward L. Fink, Department 
of Communication Arts and Theatre.) 


On Practical Applications of the ITRULE Algorithm 

Rodney M. Goodman and Padhraic Smyth Department of Electrical Engineering. Caltech 116-81, 
Pasadena, CA 91125, and Communication Systems Research, JPL 238-420, Pasadena, CA 91109 

In a previous paper we described the ITRULE learning algorithm which derives the most 
informative set of probabilistic rales from a set of sample data. The algorithm uses information-theoretic 
bounds to optimally constrain its search in the exponentially large space of possible rales. Here we 
describe our most recent work on the algorithm. We define the classes of problems which require such 
general sets of rules rather than more specific solutions such as decision trees. Typically these problems 
involve partial and missing information, initial context information, and require joint probability 
estimates over arbitrary subsets of domain variables. We describe the application of ITRULE to 
classification problems of this nature, using real data such as medical databases and congressional voting 
records. We also desenbe how ITRULE can be used as a knowledge acquisition tool to extract rales from 
databases where no domain experts exist. In particular, we describe how the algorithm can infer fault 
diagnosis and situation assessment rales via causal simulations of complex man-made systems which 
have not yet been built, for example, subsystems of NASA’s planned space station and JPL’s Mars rover. 


A Dual Control Strategy to Minimize the Discrimination Information for Stochastic Systems 
Charles D. Schapcr, Duncan A, Mcllichamp, and Dale E. Scborg Department of Chemical and Nuclear 
Engineering, University of California, Santa Barbara, CA 93106 

A new dual control policy is synthesized to minimize the disenmination information associated 
with a stochastic system and a nonstationary Gaussian-distnbuted performance specification. The system 
is characterized by a linear AR model with stochastic coefficients. The first and second moments of these 
stochastic coefficients are estimated recursively by a Kalman Filter. To indicate the presence of a dual 
effect, the mutual information measure is employed to quantify the statistical relationship between the 
estimated AR model coefficients and the available data. A parametric analysis of the mutual information 
measure is conducted to prove that the new stochastic control strategy possesses an inherent dual effect in 
terms of the fundamental parameters of the control system synthesis problem; whereas previous dual 
control strategies have had to incorporate an ad hoc weighting coefficient to obtain the dual effect. The 
relationship of the proposed dual control strategy with suboptimal duai policies is established via 
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theoretical analysis and simulation. Hence, an important link between information theory and systems 
theory is established through this new information theoretic control strategy for stochastic systems. 


On the Autocorrelation Functions of Binary Sequences Obtained from Finite Geometries 
Agnes Hui Chan, Andrew Klapper, and Mark Go resky College of Computer Science, Northeastern 
University, 360 Huntington Ave., Boston, MA 02115 

Maximum period linear feedback shift register sequences with non-linear feedforward functions 
have been used in modem communication systems. Many of these sequences are required to have high 
linear complexities and good autocorrelation function values. Recently, Chan and Games introduced a 
class of binary sequences obtained from finite geometries using nonlinear feedforward function 
p:GF(q)-*GF( 2), with q odd. They showed that these sequences have high linear complexities. 
Brynielsson had studied similar problem with q even and established the linear complexities of these 
sequences in terms of the polynomial expression of the function p. In this paper, we consider the 
autocorrelation functions of these sequences, and established their values in terms of the autocorrelation 
va' les of the sequence obtained from (p(fl°),p(f)), ■ ■ • , p (/3 , * J )), where /) is a primitive element of 
GF(q). 


Digital Synchronous Processes Generated by a Stationary and Independent Symbol Sequence- 
General Properties 

Adolfo V. T. Cartaxo and Augusto A. de Albuquerque Instttuto Superior Tecnico (1ST), DEECICAPS, 
Av. Rovisco Pais, P-1096 Lisboa, Portugal 

We study the digital synchronous processes (DSP) generated by a stationary and independent 
symbol sequence The contnbuuons of this paper arc: l) the calculation of a general expression for the mh 
order moment generating funcuon (MGF) and the conclusion that these processes arc strict-sense 
cyclostationary (SSCS); li) from the mh order MGF,, the calculation of general expressions for the 
expected value, autocovariance, autocorrelation and power spectral density (PSD) cf these processes; iii) 
from this theory, the computation of the PSD of a digital pulse position modulation (DPPM) system. 


Existence, Construction Methods and Enumeration of Higher Dimensional Hadamard Matrices 
Yang Yi Xian PO Box 145, Dept of Inform. Eng., Beijing University of Posts and Telecomm. PH. China 
The following open problems (appeared in Trans. IEEE IT-25, 566-572, 1979) are solved in this 
paper (a)' A few conjectures are proved, (b). many new construction methods are shown, (c): the upper 
and lower bounds for the enumeration arc proposed, (d). there exist no absolutely improper n-dimensional 
Hadamaid matrices of order 2 (e): there arc 4128 4-dimcnsional Hadamard matnccs of order 2. 
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Decoding Cyclic and BCH Codes up to the Hartmann-Tzeng and Roos Bounds 

G. L. Feng and K. K. Tzeng Department of Computer Science and Electrical Engineering, Lehigh 

University, Bethlehem, PA 18015 (40 mia) 

There are many cyclic and BCH codes having their minimum distance lower bounded by the 
Hartmann-Tzeng (HT) or the Roos bound. In decoding such codes, multiple syndrome sequences are 
available. In this paper, the application of multisequence shift-register synthesis algorithms - a 
generalized euclidean algorithm and a generalized Beriekamp-Massey algorithm - to decode such codes 
up to the HT bound and the Ros bound is formally considered. The main task is to determine under what 
condition or conditions the connection polynomial of the shortest linear feedback shift-register obtained 
by the algorithms will be the error-locator polynomial. We give a detailed treatment on the application of 
the geneialized Euclidean algorithm only since the application of the generalized Beriekamp-Massey 
algorithm follows similariy. For decoding up to the HT bound, we have shown that the algorithms will 
always produce the error-locator polynomial. However, for decoding up to the Roos bound, this is not the 
case unless additional conditions are satisfied. For cases that these conditions are not satisfied, the 
syndrome sequences are shown to be linearly dependent. Based on this dependence relation, an 
alternative decoding procedure is derived. 


Pseudocyclic (n,k) MDS Codes over GF(q) 

Arvind Krishna and Dilip V. Sarwate Coordinated Science Laboratory and the Department of Electrical 
and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana IL 61801 

We consider the existence of nontrivial pseudocyclic (n,k) maxtmum-distance-scparable (MDS) 
codes modulo Or* -a) over GF(q). When n is a divisor of q + 1, such codes exist for all k if n is odd, but 
if n is even, such codes exist only for odd k (only for even k) whenever o is (is not) a quadratic residue in 
GF(q). When n is a divisor of q -1, pseudocyclic MDS codes exist if and only if the multiplicative order 
of a is a divisor of ( q-])ln . (This research was supported by the U.S. Army Research Office under 
contracts DAAG29-84-K-0088 and DAAL03-87-K-0097.) 

Quasi-Cydic Codes on the Klein Quartic over Gr (2'): A Procedure for Correcting 1 or 2 Errors. 

A. Thiong-Ly Dept, of Mathematics, University of Toulouse Le Mirail, 5, Alle'es Antonio Machado, 
31058 Toulouse, France 

Let r aO mod 3. We construct a class of quasi-cyclic codes over GF(2 , ‘ denved from the Klein 
quartic: X^Y+Y^Z + Z^X =0. An easy procedure for correcting one or two errors is proposed, which 
needs at most 200 products in the finite field GF(2'). 


Generalized Remainder Decoding Algorithm for Reed-Solomon Codes 

Masakatu Moni and Masao Kasahara Department of Electronics and Information Science, Kyoto 
Institute of Technology, MatsugasaJa, Sakyoku, Kyoto, 606 Japan 

Investigating efficient decoding algonthms of Reed-Solomon codes, whose minimum distance is 
large, is very important from both theoretical and pracucal points of view. In 1982, E. R. Beriekamp and 
L Welch proposed a new decoding algomhm for RS TCH codes without computing the syndromes. It is 
called remainder decoding algorithm. It is very interesting that its key-equation is quite different from 
that of conventional algonthms; for example, Peterson algorithm, Beriekamp-Massey algonthm and 
Euclidean (Sugiyama-Kasahara-Hirasawa-Namckawa, SKHN) algomhm 
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In this paper we shall present another type of the remainder-decoding algorithm. Furthermore we 
shall give very useful properties of our generalized algorithm for decoding Reed-Solomon codes as fast as 
possible. 


Systematic Decoding of Reed-Solomon Codes 

Ron M. Roth and Abraham Lempel Department of Computer Science, Technion - Israel Institute of 
Technology, Haifa 32000, Israel 

An rxr matrix A = is called circulant if a- t , = aoj-, (indices taken modulo r). In many 

cases, we can transform the parity-check matrix of an [n,n-r] Reed-Solomon tRS) code into 
H * [A A ‘~ 2 ■■■At], forming a concatenation of several circulant matrices. A decoding procedure for 
RS codes is derived, based on such a representation of the parity-check matrix. The key idea in the 
proposed algorithm is a transformation of the Beriekamp-Massey algorithm into the time-domain using 
an /--dimensional inverse Fourier transform, compared with the /t-dimensional transform used in Blahut’s 
decoder. The decoding algorithm consists of the following steps: (i) syndrome evaluation, using the 
encoding circuit; (ii) finding the error-locator and error-evaluator polynomials using a reduced version of 
Blahut’s decoder; and (iii) interpolation of the outputs of Step (ii), reusing the encoding circuit The 
resulting procedure inherits both the (relatively low) time complexity of the Beriekamp-Massey 
algorithm, and the hardware simplicity charactenstic of Blahut’s time-domain algonthm. In particular, 
the required memory size is proportional to r, rather to n. 


The Cannibalistic Traits of Reed-Solomon Codes 

Oliver Collins Department of Electrical and Computer Engineering, Johns Hopkins University ,, 
Baltimore, MD 2/2/S 

This paper presents a new kind of algebraic code formed by combining Reed-Solomon codes with 
themselves. The concatenation scheme allows longer code woids than Reed-Solomon codes wiih the 
same symbol set. Examples of the performance gains with a constraint length 15, maximum likelihood 
decoded, convolutional inner code appear at the end. Although the goal of coming as close as possible to 
channel capacity on the Gaussian channel drove the design of the codes presented, they are also suitable 
for the photon channel and, to a lesser extent, the magneue recording channel. 


Decoding of Reed-Solomon Codes Using Bit Level Soft Decision Information 

Alexander Vardy and Yair Be’ery Department of Electronic Communications, Control and Computer 

Systems, Tel Aviv University. Ramat Aviv 69978, Tel Aviv, Israel 

In this paper we present a Reed-Solomon decoder that makes use of bit soft-decision information. A 
Reed-Solomon generator matrix which possesses a certain inherent structure in GF( 2) is derived. This 
structure enables representation of the code as a union of coscts, each coset being an interleaver of several 
binary BCH codes. Such partition into coscts provides a clue for efficient bit level soft decision decoding 
The proposed decoding algonthms are several orders of magnitude more efficient than conventional 
techniques for many codes 
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Noise Effects on Af-ary PSK Trellis Codes 

Gideon Kaplan and Ephraim Zehavi Ministry of Defence, Tel-Aviv, Israel and Department of Electrical 
Engineering, Technion, Israel Institute of Technology, Technion City, Haifa, 3200, Israel, and 
Qualcomm, Inc, 10555 Sorrento Valley Road, San Diego, CA 92121 

In this work we present upper bounds on the error performance of M- ary PSK trellis coded system 
over the AWGN and in the presence of Tikhonov distributed phase noise, this model is applicable to a 
first-order phase tracking loop perturbed by thermal and earner phase noise. 

Our model is as follows. A binary sequence at the transmitter is encoded using a rate R = (n - 1)1 n 
trellis code having an S-state encoder. The n-tuple produces one of M = 2" PSK signals. The channel 
produces at the output a noisy discrete-time sequence [y p rcurl,y p = ppX p + n p . Here, p p = exp(jO p ) is a 
unit vector possessing a Tikhonov distnbution. It is assumed that the phase noise process is narrowband 
with respect to the data rate. An ideal interleaver/deinterleaver system is also assumed. We are interested 
in the error performance of this coded system. 

Using Chemoff bounding techniques the pairwise error probability of the decoder, which uses the 
squared Euclidean distance as its metric, is estimated This bound is then used to calculate the 
"generalized Ro". the degradation due to phase noise is demonstrated for 8-PSK and 16-PSK. 

The error perfoimance of some practical coding methods for 8-PSK and 16-PSK are also presented, 
based on the modified generating function approach. It is shown that there exists some codes which are 
more robust for a given level of phase noise, and that these codes have parallel transitions in their trellis, 
thus their minimum Euclidean distance is not ncccssanly optimal. However, they exhibit a greater degree 
of robustness to phase noise than the optimal codes for the AWGN channel 


Bidirectional Trellis Decoding 

Farhad Hcmmati Comsat Laboratories, 22300 Comsat Drive, Clarksburg, MD 20871 

Bidirectional trellis decoding is a reduced-complexity method for soft-decision decoding of block 
codes. Unlike the Viterbi aigontbn, which processes every trellis stale, the bidirectional decoding 
algonthm takes advantage of the structure of the considered code to identify and process a small subset of 
paths in the trellis diagram containing the most likely path. The algorithm examines a received block of 
channel symbols in the forward and in the backward direction and selects the most likely codeword as the 
transmitted message. The bidirectional algorithm takes significantly fewer computations per decoded bit. 
For example, it takes 647 binary operations for decoding the extended Golay code, whereas Forney’s 
decoding method requires 1351 binary operations. Extensive analysis and computer simulations indicated 
that for special classes of block codes, including the extended Golay code, the BER performance of die 
bidirectional decoding algorithm is equivalent to that with a maximum-likelihood decoder. 


Rotationally Invariant Trellis Codes 

Steven S. Pietrobon, Daniel J. Costello, Jr., and Gottfried Ungetbocck Department of Electrical and 
Computer Engineering, University of Notre Dame, Notre Dame, IN 46556, and IBM Research Division, 
Zurich Research Laboratory, Saumerstrasse 4, CH-8803 Ruschlikon, Switzerland 

A general parity check equation for rotationally invariant rate kl(k+ 1) trellis codes is presented. 
This equation is different from the traditional linear parity check equations (which do not give 
rotationally invariant codes) in that modulo M (Af > 2) anthmclic is used to derive the parity check 
equation. The signal set mapping also needs to be related to modulo-Af anthmeue in terms of its 
rotational properties (e.g., MPSK and other "naturally mapped" signal sets) 
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The final parity check equation still uses modulo-2 addition for the binary sequences, but non-linear 
terms are added into the equation. These non-linear terms overcome the effect of a phase rotation. In 
contrast to the linear codes, the encoders denved from the non-linear parity check equation may not be 
minimal. 

Using this parity check equation, rotationally invariant rate 2/3 trellis codes with 8PSK modulation 
are presented. These codes exhibit a coding gain slightly inferior to linear codes, but they have more 
tolerance to phase slips within the demodulator. (This work was supported by NASA grant NAG5-557 
and by NSF grant NCR89-03429.) 


Trellis Coding using Multi-Dimensional QAM Signal Sets 

Steven S. Pietrobon and Daniel J. Costello, Jr. Department of Electrical and Computer Engineering, 
University of Notre Dame, Notre Dame, IN 46556 

A method of finding good trellis codes with multi-dimensional (multi-D) QAM modulation is 
presented. Using the 16QAM signal set, 4-D, 6-D, and 8-D QAM signal sets are constructed which have 
good partition and phase rotational properties. 

The good partition properties are achieved by the use of block codes and their cosets restricting each 
level in the multi-D mapping. The rotational properties are achieved through the use of a "naturally 
mapped" 16 QALM signal set. This signal set has the property that, of the four bits used to map the signal 
set, only two bits are affected by 3 90° phase rotation. With an appropriate addition of the coset 
generators, the multi-D signal sets also have two mapping bits affected by a 90° phase rotation (the 
remaining bits being unaffected. 

This implies that many good rate k/(k + 1) trellis codes can be found for effective rates between 3.0 
and 3.75 bit/T and that 90° or 180° transparent. The results from a systematic code search using these 
signal sets are presented. (This work was supported by NASA grant NAG5-557 and by NSF grant 
NCR89-03429.) 


The Extended-D£5: a Trellis-based Attack Strategy 

Jorge M. N. Pereira Centro de Analtse e Processamento de Sineus, Departamento de Engenharia 
Electrote'cnica <t Computadores, Instituto Superior Te'cnico, 1096 Lisboa Codex, Portugal 

In previous work a new and powerful ciphering method based on the DES was suggested adding, 
through 2 Control Bit Streams, a further sccumy level (Extended-D£S-E DES). Later it was recognized 
that, given the Key, it would be very easy to recover the Plaintext in spite of the additional security. A 
simple protection strategy was devised consisting of pre-shifts of the Plaintext and Ciphertext (EE DES). 
We now describe the attack Strategy (using a trellis approach), and show the sccumy provided by the 
protected strategy referred to above. Let it be noticed that the additional protection can be controlled by 
means of the Control Bit Streams' generation (Pseudo-Noise Bit Sequences) and of the slowly varying 
pre-shifting Control Words. Considcnng that both the E DES and the EE DES can be implemented with 
a minimum investment and complexity upon the existing DES components, the new ciphering system is 
simple to use remaining extremely versatile. 


Performance of Trellis Coded Run-Length Codes 

Mignon Belongie and Chris Hccgard School of Electrical Engineering, Cornell University, Phillips Hall, 
Ithaca, NY 14855 

This paper concerns the performance a family of codes which combines the error correcting 
capabilities of convolutional codes with run-length constraints. These codes have application to magnetic 
and optical recording systems. 

First, trellis constrained, run-length codes (TCRLCs) are described The combination of a cosct 
code, C, and (d, k) parameters induce a combined constraint on allowed signals. A signal is a codeword 
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of the TCRLC if and only if: (1) the sequence of cosets determined by its transition times, r„, is the coset 
code. C, and (2) the tun-lengths satisfy the given (d, k) run-length constraint, (d + 1) A £ 7". S (k + 1)A 
where the n'* run-length T„=t n - 1 „,i and A is the clock period. 

After TCRLCs are described, the problem of decoding will be addressed. The decoding problem 
involves a three step procedure: (1) Detection, (2) Sequence Estimation and (3) "Un-encode”. The second 
procedure, in the case of a recording channel, must make an estimate in the presence of extensive channel 
memory. This step, which is the most difficult to develop, must trade sequence estimation performance 
for complexity. 

A study is made of the performance of TCRLCs with the described detecuon methods. In particular, 
an example the performance of a code that dominates the industry popular "2,7" and ”1,7" codes (which 
we tenn "The Sevens Killer Code") is presented. 


Low-Complexity Maximum-Likelihood Decoding Algorithm for Non-Binary Trellis Cod»s 
Gareguin S. Markarian and Haik H. Manukian Radiophysics & Electronics Institute, Armenian Academy 
of Sciences, 378410, Ashtarak-2, Armenia, USSR 

In this paper we propose the new simple and fast maximum likelihood decoding algorithm, adapted 
for non-binary balanced (dc-free) trellis codes. These codes represent a class of codes which map one 
binary bit into one <?-ary (q > 2) channel symbol and are used in digital transmission systems, operating 
by cable line. In order to explain the proposed algorithm we apply the most widely used ternary (q = 3) 
bipolar or the "alternate matk inversion" (/All) balanced trellis code. The positive effect is achieved by 
expanding the channel alphabet q to infinity and introducing interdependencies between the channel 
symbols such that not all vectors of length n with components from expanding alphabet are suitable. 

The structure of these codes allows to construct simple and fast soft Vitcrbi decoder without 
analog-to-digital converters. In order to interpret the proposed algorithm we use the most widely used 
ternary trellis "bipolar" or "alternate mark inversion” (AMI) code as an example. The proposed algorithm 
allows to correct some errors in codes without error correcting properties. The new upper bound for 
probability of error per symbol is also proposed in this paper. We have found that according to proposed 
algorithm a theoretical energy gain of 3 5 dB as compared with symbol-by-symbol hard decision 
detection in AWGN channel is possible. Finally, we present the realization of low-complexity soft Viterbi 
decoder which proves the operation ability of proposed algorithm. 


Performance Evaluation of Trellis Coded Modulations with Memory 

Witold Holubowicz and Fidel Morales-Moreno Technical University of Poznan, 60-965 Poznan, Poland, 
and Telesat Canada, Gloucester, Ontario, KIB 5P43 Canada 

This paper presents some tools useful in performance evaluation of coded modulations with 
memory. In the first part of this paper, an efficient method is presented for finding the best combinations 
of coded modulations with memory, in the sense of maximization of the minimum Euclidean distance 
dm*, without the necessity of doing an exhaustive search. If we want to maximize d mm for a class of 
coded modulauons, the search must in general go over all modulations and all codes of interest. In our 
procedure, used for the optimization of coded Correlative FM signals, we first calculate the upper and 
lower bounds on the for any input sequence with given weight m, for the modulator of Correlative 
FM without any code, then, we separate all convolutional codes of interest into subclasses, each subclass 
described by its minimum Hamming distance values, finally, using the previously calculated d m „ bounds, 
our search may be limited to one or two subclasses of codes only, stiU with the same results as those of an 
exhaustive search. 

The second pan of our paper deals with ways of calculating upper bounds of the BER for coded 
modulations with memory. Calculauon of the upper bound for a coded memoryless modulation requires, 
in general, inversion of matrices of size N2xN2 where N is the number of encoder states. This becomes 
impractical even for coded modulauons without memory. Zehavi and Wolf proposed a method to 
calculate this bound using only matrices of size N x At, the method however was applicable only to coded 
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modulations characterized by: 1) linear encoder, 2) code rate R = (n-l)/n; 3) memoryless modulation; 4) 
coded modulations with, so called, "uniform etror profiles”. 

In our paper, we generalize the method described in Zehavi and Wolf by relaxing, step by step, 
restnctions 2), 3) and 4). In the first two cases, the resulting number of states are still of size N xN. When 
the etror profiles are not uniform, the resulting matrix is larger than N x N, but it is still much smaller than 
N 2 xN 2 . We show simple examples of coded TFM and coded Duobinary MSK, with uniform and non- 
uniform error profiles, which illustrate the procedures presented earlier. 


Multi-Level Multidimensional Trellis Codes 

Jiantian Wu and Xuelong Zhu Department of Electronic Engineering, Tsinghua University, Beijing, PJi 
Of China 

Recently the multi-level codes which allow the use of suboptimal multi-stage decoding procedures 
have well been studied. In this paper, we describe a design method for multi-level codes based on 
multidimensional signal constellations. We show how to decrease decoding complexity and error 
coefficients (multiplicity) We present a number of multi-level trellis codes based on four- and eight¬ 
dimensional two-way lattice partitions, which have much higher fundamental coding gains than 
previously known coset codes with moderate decoding complexity. The simulation results of a simple 
code show that the new code has about the same performance/complexity tradeoff as the previously 
known best coset codes. 
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PLENARY SESSION 
Thursday, 8 - 8:50 a.m. 


Trying to Beat the Heisenberg Principle, 

Alberto Gninbaum, Department of Mathematics, University of California, Berkeley, CA 94720 
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TECHNICAL SESSIONS 


Thursday, 9 a.m. -12 m. 

SESSION ThAl 
ESTIMATION H 


Why Least Squares and Maximum Entropy? An Axiomatic Approach to Inverse Problems 
Imre Csiszir Mathematical Institute of the Hungarian Academy of Sciences, Budapest, P.OJ3. 127, H- 
1364, Hungary (40 mm) 

For the set S of all real-valued or all probability mass functions with a given finite domain, all 
conceivable rules for selecting an element of a feasible subset of S, determined by linear constraints, are 
considered, and those satisfying certain natural postulates are characterized. Two basic postulates imply 
that the selection should minimize some function defined on S which, if a prior guess is available, is a 
measure of distance from the latter. It is shown how invariance properties and a transitivity postulate 
restrict the class of permissible distances, leading to characterizations of some well-known families of 
distances and also some new ones As corollanes, unique characterizations of the methods of least squares 
and minimum discnmination information are amved at. The latter are uniquely characterized also by a 
postulate of composition consistence. As a special case, a unique characterization of the method of 
maximum entropy from a small set of natural axioms is obtained. 


A Method of Sieves for Regularizing Maximum-Likelihood Spectrum Estimates 
P Moulin, D. L. Snyder, and J. A. O’Sullivan Electronic Systems and Signals Research Laboratory, 
Department of Electrical Engineering, Washington University, St Louis, MO 63130 

Maximum-likelihood (ML) spectrum estimation is a notonous ill-posed problem. In this paper, we 
are concerned with the use of a new rcgulanzahon method for addressing this fundamental issue. We 
recommend a method of sieves, based upon the following concepts. The spectrum belongs to a subset of 
some Hilbert space of functions over which a complete set of nonorthogonal basis functions is defined. 
The spectrum is then represented by a countable set of coefficients in a nonorthogonal senes expansion. 
By defining an appropnate sieve on this countable set, our problem reduces to maximum-likelihood 
estimauon of the parameters in the sieve. Three main attractive features of this approach are: (1) the 
nonorthogonal expansion is a convenient framework for defining the sieve and including a priori 
information, (2) mean-square consistence of the estimates can be expected; and (3) we have derived a 
tractable alternating maximization algorithm for estimating the parameters. The setup of this problem is 
very general and can be applied without major difficulties to the estimation of higher-dimensional 
spectral functions. 


The Index of Resolvability of Probability Density Estimators 

Andrew R. Barron Departments of Statistics and Electrical <t Computer Eng , University of Illinois, 725 
S, Wright Street, Champaign, IL 61820 

An index of resoivability is defined which bounds the rate of convergence of density estimators 
based on the mmimum dcscnption length (MDL) principle due to Rissancn, Wallace, So-km, and others. 
Given the sample size n, a countable collection T„ of probability densities, and codelengths L,(q), q s r„ 
which sausfy Kraft s inequality, the estimator is defined as the minimizer of the total description 
length LJq)+ log Vq(X'), q e F. The index of "solvability of a density p relative to T, and is 
defined by 
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R„(p) = min «-- + D(p\\q) ► 

*er. /: 

v * 

where D (p||<?) is the relative entropy. This index is reen to bound the redundancy of a universal noiseless 
source code based on the estimator. Also, it bounds the rate of convergence in squared Hellingcr distance, 
i.e.. 


d 2 (P'Pji-'O pr (R a <p)). 

Bounds on the resolvability are determined in parametric and nonparametric cases, yielding near 
optimum rates of convergence. For instance, estimators based on sequences of approximating exponential 
families, with the order selected by the MDL criterion, are shown to converge at rate 0 /r (/i -S ' /(2r+1) log n ) 
for log-densities with r squarc-integrablc derivatives and Op ,(log n)l n for densities in one of the families, 
without prior knowledge of whether the density is finite or infinite dimensional. 


On Estimation of Discrete Hammers! ein Systems by the Fourier and Her mite Series Estimates 
Adam Krzyzak Department of Computer Science Concordia University, 1455 De Maisonneuve Blvd. 
West, Montreal, Canada H3G 1M8 

We study the estimation of a single-input, single-output (SISO) discrete Hammerstein system. Such 
a system consists of a nonlinear, memoryless subsystem followed by a dynamic, linear subsystem. We 
identify the parameters of the dynamic, linear subsystem. We identify the parameters of the dynamic, 
linear subsystem by the standard correlation and Newton-Gauss method. The main results concern the 
estimation of the nonlinear, memoryless subsystem, recovering the nonlinearity using the Fourier and 
Hcrmitc series regression estimates. We prove the density-free pointwisc convergence of the estimates, 
that is the estimates converge for all input densities. The rates of pointwisc convergence are obtained for 
smooth input densities and for nonlinearities of Upschitz type. Global convergence and its rate arc also 
studied for a large class of nonlinearitics and input densities. 


On Estimation of Hammerstein Systems by the Recursive Kernel Regression Estimate 

Adam Krzyzak Department of Computer Science, Concordia University, 1455 De Maisonneuve Blvd. 
West. Montreal. Canada H3G 1M8 

In this paper we study the estimation of multi-input, single-output discrete Hammerstein system. 
Such a system contains a nonlinear, memoryless subsystem followed by a dynamic, linear subsystem. We 
obtain the impulse response of the dynamic, linear subsystem by the correlation method. We as well 
estimate coefficients of the ARMA model describing the linear subsystem. The main results concern the 
estimation of the nonlinear, memoryless subsystem. We impose no conditions on the functional form of 
the nonlinear subsystem, recovering the nonlinearity using the recursive kernel regression estimate. We 
prove the distribution-free pointwisc and global convergence of the estimate, that is, no conditions arc 
imposed on the input distribution and convergence is proven for virtually all nonlinearities. The rates of 
pointwise as well as global convergence are obtained for all input distributions and for nonlinearities of 
Lipschitz type. We also discuss possible applications of the studied nonlinear systems in detection and 
adaptive control. 


A New Lower Bound of Cram£r-Rao Type for Quantum State Estimation 
Hiroshi Nagaoka Faculty of Engineering, Hokkaido University. Sapporo, Japan 

Let H be a Hilbert space, and let S = {Sg.0 e 9 c JR") be a family of quantum states on H 
smoothly parameterized by 9 =(0 I - We consider the parameter estimation problem for S. A 

quantity CR e f? depending on 6 is called a lower bound of Cramer Rao type if the covariance matrix 
V € It 1 *" of any unbiased estimator at the state S$ always satisfies Tr GV £ CR, where G e #?**" is 
given positive-definite weight matrix. Well-known bounds are CRs based on symmetric logarithmic 
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derivatives and CR r based on right logarithmic derivatives. A. S. Holevo constructed another bound CR H 
such that CR h 2 max (CX$ Cr R ). We introduce a new bound CR NEW for the case n = 2, which is based 
on a fundamental inequality on simultaneous measurements of two noncommutative observables, and 
show that CRf/ny 2 CRh. CRhew is defined via some minimization that cannot be solved explicitly in 
general. In the simplest noncommutative case dim H = 2, however, it is explicitly written as 
CRhew = CR S + (dclGIdelJ)' A Tr Abs L 2 ], where {i- 1 _ L 2 ) are the symmetric logarithmic 

derivatives of S at 8 and J = [J tJ ] is a 2x2 matrix such that'/,, =ReTiS a L,Lj. Moreover CR NEW is 
shown to be the maximum among all the lower bounds of Cramdr-Rao type in this case. 


Asymptotic and Geometric Procedures for Estimating Correlation and Ambiguity Functions 

Edward L. Titlebaum and Sanjay K. Mehta Department cf Electrical Engineering, University of 
Rochester, Rochester, NY 14627 

In the first section of this paper the method of stationary phase is used to derive an expression for 
the asymptotic approximation for the crosscorrelation function of two FM signals. The instantaneous 
frequency curves of these FM signals intersect at only one point in the time-frequency space. In the next 
section, for signal which are slow varying in amplitude, we derive a simple geometric interpretation for 
the asymptotic results derived in section 2. The intuitively pleasing result is that the crosscorrelation 
function between the two FM signals is shown asymptotically to be the square root of an area measured in 
time-frequency space. This result allows for quick estimation of Correlation and Ambiguity functions. 


Statistical Performances of Several Eigen-Structure DOA Estimation Methods 
Luo Jingqing and Bao Zheng Electronic Engineering Institute, Xidain Umversity, Xi’an, P. R. China 
This paper presents some asymptotic statistical performances of several eigen-structure DOA 
estimation methods, named MUSIC method, Minimum-Norm method, Minimum eigenvector method, 
and Johnson eigenvector method, from the biases and vanances of the parameter estimations. The concept 
of unbiased signal subspace estimation is introduced. The conclusion is that when the noises are zero 
mean Gaussian white noises and independent of the signals, unbiased signal subspace estimation is 
obtained, and all four methods presented above give an unbiased parameter estimation, but their variances 
are different. The MUSIC method and Johnson eigenvector method have the smallest variances, and the 
Minimum eigenvector method has the largest vanance. Some of these results conflict with the results of 
other authors. Calculations and graphs are presented to show these properties. 
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SESSION ThA2 


MULTIPLE ACCESS HI 


A Lower Bound to the Packet Waiting Times in the Infinite Population Multiaccess Channel 
Mart L. Molle Computer Systems Research Institute, University of Toronto, Canada M5S1A4 

Just as recent analyses of multiaccess conflict resolution algorithms have progressed from (merely) 
finding their maximum achievable throughput to finding complete delay-throughput response time curves, 
we now extend the wotk on algorithm-independent upper bounds to capacity by giving a lower bound to 
the mean delay-throughput curve. Using a "helpful genie” approach, we reduce the problem from a 
distributed multiaccess system (where individual packet arrival times are not visible to the algorithm) to a 
centralized single server queueing system where bulk service (i.e., group testing) is permitted. These 
customer arrival times consist of the "real” packet arrival times, augmented with "dummy” points from an 
independent Poisson process at rate A(l/p-l). Thus, customer arrivals form a Poisson process at rate 
Xlp, and each customer represents a "real” packet according to an i.i.d. Bernoulli trial with probability p. 
For p sufficiently large, the (bulk) service discipline that minimizes the mean waiting time in the genie- 
aidcd queueing system is easily found using known results for optimal group testing algorithms under the 
Bernoulli arrival sequence model. 


Lower Bound for Packet Delay in Random Multiple Access System 

B. S. Tsybakov and N. B. Likhanov Institute for Problems of Information Transmission, Academy of 
Sciences of the USSR, 19 Ermolova str GSP-4, Moscow 101447. USSR 

We consider a packet communications network using a chaitnel with slotted random multiple access 
(RMA) and ternary feedback {0, = 0 empty, 0, = I success, 0, = 2 conflict) about event in slot [r, r+ 1). 
It is assumed that there is an infinite number of users and the overall flow of requests for packet 
transmission is Poisson one with intensity A. 

Here the RMA algorithm is an initial set B 0 C (0, «■) and a function 0, = B,(0 O ,.... 0,_i) defined over 
all 0,e (0,1,2), OSi Sr-1, r=0,1, • • • and taking values of sets B, from time interval [0,»). A 
rcady-for-transmission packet is transmitted in slot (r, t + 1) if and only if its moment of generation x is in 
0,. The packet delay d = d (A) is defined for a given algorithm as the sum of the mean delays of packets 
generated in slot (r, r + 1) and divided by A with t -> ~. 

The stated problem is to find a lower bound d(A) [for d( A)) that is true for all algorithms. The 
problem is more general than the upper-bound problem for capacity of an RMA system, since the value of 
minimum intensity A = A 0 (such that d(A 0 ) = *>) gives us the upper bound C = A<>. 

We find a lower bound d(A) and compare it with the mean packet delay for the part-and-try 
algonthm. 


Theory of Packet Reservation Multiple Access 

David J. Goodman, Sanjiv Nanda, and Uzi Timor Wireless Information Networks Laboratory, ECE 
Department, Rutgers University, Box 909, Piscataway, NJ 08855-0909, and Ministry of Defense, Haifa, 
Israel 

Packet Reservation Multiple Access (PRMA) makes it possible for wireless voice terminals to share 
a short range radio channel. As a statistical multiplexer, PRMA is subject to fluctuating packet rates from 
the ensemble of terminals sharing the channel. Because voice packets require prompt delivery, PRMA 
responds to congestion by dropping packets delayed beyond a specified time limn. 

In this paper, we model the ensemble of terminals as a set of coupled queues and use equilibrium 
point analysis to evaluate system behavior. Based on the cquilibnum values of system state variables, we 
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derive the probability of packet dropping as a function of the system variables, including design 
parameters and operating conditions. We also establish conditions for system stability and efficiency. 

Numerical calculations based on the theory show close agreement with published results based on 
computer simulations. They also provide valuable guides to system design. 


An Architecture for Very High Speed Packet Switching Systems 

R. L. Cruz Dept, of Electrical <& Computer Engineering, University of California at San Diego, La Jolla, 
CA 92093 

We consider a class of architectures for very high speed packet switches that is based on the concept 
of error correcting codes. 

The inputs and outputs of a switch are labeled by binary strings of length n. The outputs are 
partitioned into output trunk groups. The function of the switch is to route packets from the inputs to the 
correct output trunk group. Each output trunk group is assigned a binary string of length n, called a 
codeword. The switch attempts to route a packet to the output whose label is the codeword assigned to the 
desired output trunk group Misdirection may occur due to contention; this corresponds to "errors" 
occumng in the "transmitted codeword". By appropriately partitioning the outputs into output trunk 
groups, we find that a very reliable and fault tolerant mechanism for routing packets may result 

We consider a specific switch design based on a bufferless omega network. For a simple model for 
packet arrivals, we find that the optimal partitioning does not correspond to minimum distance decoding 
with the Hamming metric; in fact the optimal partitioning depends on the utilization of the switch. 


Stochastic Monotonicity Properties of Multi-Server Queues with Impatient Customers 

Partita P. Bhattacharya and Anthony Ephremides Electrical Engineering Department & Systems 

Research Center. University of Maryland, College Park. MD 20742 

We consider mulh-servcr queues in which a customer is lost whenever its waiting time is larger than 
its (possibly random) deadline. For such systems, important performance measures are the number of 
(successful) departures and the number of lost customers over a lime interval. We establish sufficient 
condiuons on the arrival, service, and deadline processes and on the number of servers, for strong 
stochastic order monotonicity of these measures as funcUons of some of the system parameters. 


The IFFO Protocols Revisited: An Extension for Integrated Communications 
Jeffrey E. Wieselthier and Anthony Ephremides Information Technology Division, Naval Research 
Laboratory, Washington, DC 20375, and Electrical Engineering Department, University of Maryland, 
College Park, MD 20742 

A class of multiple-access protocols for data traffic, known as the Interleaved-Frame Flush Out 
(IFFO) protocols, were first introduced in 1980. These protocols were shown to be well suited for satellite 
communication environments. In this paper we show how the IFFO protocols can be modified to 
accommodate a mixture of voice and data traffic by incorporating a "movable-boundary" mechanism. We 
desenbe the analytical model of the protocols in detail. We present exact and approximate methods of 
analysis that are based on the determination of the equilibrium behavior of infinite Markov chains, and 
which lead to accurate performance evaluation, or in some cases to bounds on performance. Questions of 
optimizaUon of bandwidth allocation to the two classes of traffic are also addressed. 
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Conflict Resolution Algorithms for High Error-Rate Multi-Access Channels 
George C. Polyzos and Mart L. Molle Dept, of Computer Science 4 Engineering, University of 
California, San Diego, La Jolla, CA 92093-0114, and Computer Systems Research Institute University of 
Toronto, Toronto, Canada MSS 1A4 

Previous authors have investigated the performance of random-access Tree Conflict Resolution 
Algorithms for synchronous broadcast communications channels under a simple model for channel errors. 
This model, proposed by Massey and also by Vvedenskaya and Tsybakov, deals only with forward errors, 
and it assumes that stations remain synchronized and receive identical feedback. Furthermore, only the 
following two types of errors are considered: (i) a single transmission that gets destroyed by noise and is 
interpreted as a conflict, and (ii) an otherwise idle channel that, because of channel noise, tricks the 
receivers into believing that a conflict occurred. 

It has been reported that, assuming independent errors with probabilities e and S, respectively. Tree 
Algorithms with gated or windowed channel access have capacity zero for S > 0.5. However, we now 
show that with a simple modifleation, i.e„ by merely repeating slots that have been reported as conflicts 
for up to n times (or until an idle or a success is reported), we can achieve non-zero capacities for S 
arbitrarily close to 1 by choosing n 2 -1 /log2(5). Furthermore, we provide throughput-delay statistics for 
these protocols. 


The Communication Complexity of Solving a Polynomial Equation 

Zhi-Quan Luo and John N. Tsitsiklis Department of Electrical Engineering and Computer Science, 
Massachusetts Institute of Technology, Cambridge MA 02139 

We consider the problem of evaluating a function (x,y) (x e R "•y e R ") using two processors P i 
and Pi. assuming that processor Pi (respectively, Pi) has access to input x (respectively, y) and the 
functional form of /. The processors perform this computation by exchanging messages under the 
restricaon that a message sent by any processor is a continuously differentiable function of the data (x or 
y) possessed by that processor and the messages that it has already received. We establish a new general 
lower bound on the communication complexity (i.e., the minimum number of real-valued messages that 
have to be exchanged, as wcu as some general properties of optimal protocols. We then apply our results 
to the case where f(x,y) is defined as a root z of a polynomial equation 

2 (*.+?.)** =0 

i-O 

and obtain a lower bound of n (which matches the obvious upper bound). This is in contrast to the fi(l) 
lower bound obtained by applying earlier results of Abelson. Our results while very intuitive are 
surpnsingly difficult to prove and involve primarily techniques from multivariable calculus. 


SESSION ThA3 


CRYPTOGRAPHY I 


On the Quadratic Spans of De Bruijn Sequences 

Agnes Hui Chan and Richard A. Games The Mitre Corporation, Burlington Road, Bedford, MA 01730 
(40 mia) 

The quadratic span of a periodic binary sequence is defined to be the length of the shortest quadratic 
feedback shift register (FSR) that generates it. This notion generalizes the usual notion of the linear span, 
which is used to analyze the complexity of pseudorandom sequences. An algorithm for computing the 
quadratic span of a binary sequence is described. The required increase in quadratic span is determined 
for the special case of when a discrepancy occurs in a linear FSR that generates an initial portion of a 
sequence. The quadratic spans of binary De Bruijn sequences are investigated. It is shown that the 
quadratic span of a De Bruijn sequence of span n is bounded above by 2*-'An(n-l)-l and this bound 
is attained by the class of a De Bruijn sequences obtained from m-sequences. It is easy to see that a lower 
bound is fl + 1, but a lower bound of n+2 is conjectured. The distributions of quadratic spans of a De 
Bruijn sequences of span 3, 4, 5, and 6 are presented. It appears that the vast majority of de Bruijn 
sequences have quadratic spans close to the lower bound. 


Cascade Ciphers: The Importance of Being First 

Ueli M. Maurer and lames L. Massey Institute for Signal and Information Processing, Swiss Federal 
Institute of Technology, CH-8092 Zurich, Switzerland 

We consider the security of cascade ciphers where the keys of the component ciphers arc 
independent It is shown by a counterexample that the intuitive result, formally stated and proved by Even 
and Goldreich. that the cascade is at least as strong as the strongest component cipher, is only valid under 
the uninterestingly restrictive assumption that the enemy cannot exploit information about the plaintext 
statistics. It is proved that, for quire arbitrary notions of what breaking a cipher means and for any 
reasonable definition of problem difficulty, the cascade is at least as difficult to break as the first cipher. 
An obvious consequence of this result is that if the ciphers commute, then the cascade is at least as 
difficult to break as the most-difficult-to-break component cipher, i.e., the cryptographic chain is at least 
as strong as its strongest link. It is noted that additive stream ciphers do commute, and this fact is used to 
suggest a strategy for designing secure practical ciphers. 


The Hardness of Solving, with Preprocessing, Two Problems Related to Cryptography 

Antoine Lobstcin Centre National de la Recherche Scientifique, URA 820, Telecom Paris, De'partement 

informatique, 46, rue Barrault 75634, Paris 13 (France) 

The two problems Linear Decoding and Subset Sum, which have given birth to the McEliecc and 
the knapsack public-key cryptosystems, are known to be NP-complete. For the first problem, it has been 
proved that, even if one knows the linear code in advance and can preprocess it, the existence of a 
polynomial-time decoding algorithm wo"ld imply that the polynomial-ume hierarchy collapses at an 
early stage. We give a new, straightforward proof of this result, and prove that the same holds for Subset 
Sum. even if the knapsack is known in advance and can be prcproccssed, there is no polynomial-time 
algonthm solving it. unless the polynomial hierarchy collapses. 
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A CDMA Security Scheme Using Bit Inversion 

D, Despen and N. K. Huang Department of Electrical Engineering, University of Minnesota, 
Minneapolis, MN 55455 

This paper investigates a code division multiple access (CDMA) communication system security 
scheme, and evaluates the performance of two sequence types based on this scheme. The sequences to be 
investigated are the m-sequences and the No sequences. The security scheme to be studied is bit 
inversion, the intentional inverting of bits to contuse an unintended user. The two sequence types will be 
compared on the basis of effective security and the effect of bit inversion on processing gain (PG). 


Sequence Complexity and the Directed Acyclic Word Graph 

Cees J. A. Jansen and Dick E. Boekee Philips USFA B.V., P.O. Box 218, 5600 MD Eindhoven, The 
Netherlands, and Technical University of Delft, P.O. Box 5031,2600 GA Delft, The Netherlands 

Blumer’s algorithm can be used to build a Directed Acyclic Work Graph (DAWG) in linear time 
and memory from a given sequence of characters. In this paper we show that Blumer's algorithm can be 
used very effectively to determine the maximum order (or nonlinearity) complexity profile, as introduced 
by Jansen, of a given sequence in linear time and memory, where each complexity value is determined 
sequentially, i.e., after each new character of the sequence. We also show that this algorithm can be used 
to determine the penod of a periodic sequence in linear time and memory. Moreover it appears that the 
DAWG is an even more efficient means of generating the sequence, given a number of characters, than 
e.g., the non-linear feedback shift register equivalent of that sequence, as it always needs the least amount 
of characters to generate the remainder of the sequence. 


Run Permuted Sequences 

Cees J. A. Jansen and Dick E. Boekee Philips USFA B.V., P.O Box 218, 5600 MD Eindhoven, The 
Netherlands, and Technical University of Delft, P.O. Box 5031,2600 GA Delft, The Netherlands 

This paper describes the construction of classes of binary sequences, which are obtained by 
permuting the runs of zeroes and ones of some given periodic binary sequence s = (So.Ji.-.S(.-i)~, 
s, e GF(2). Based on the run-length notation of a periodic sequence a large class of sequences is 
constructed by permuting the runs of zeroes and ones of a DeBruijn sequence of given order. The 
properties of the sequences in this class, in particular the number of sequences in a class and their 
complexities, are discussed. It is shown that in this way all DeBruijn sequences of given order are 
obtained, but also many more sequences with higher complexities, all satisfying Golomb’s first and 
second randomness postulates. Hence, it is demonstrated that Golomb’s statement that the number of 
sequences in this class ”is slightly larger" than the number of DeBruijn sequences of order n, is not very 
careful. It is also shown how to generate the sequences in this class with the use of enumeralive coding 
techniques. The binary sequence generator obtained in this way can be useful for cryptographic purposes, 
e.g., in streamcipher systems. 

An Attack on the Clock Controlled Generator Sequences 

Chuan-kun Wu Department of Applied Mathematics, Xidian University, Xian, P. R. of China 

This paper investigates the complexity and statistical characterization of clock controlled generator 
sequences, shows its advantage and weakness, and gives an algebraic method to attack this model. 


The Linear Recurring Sequences over the Residue Gass Ring ZJ(m) 

Li Xiangang Dept, of Applied Mathematics, Zhengzhou Engineering & Technical Institute, P.0 Box 
1001-100, Zhengzhou, Henan, P.R. of China 

We shall deal mainly with the problems of linear recurring sequences over the residue class ring 
Z/(m), considering first die properties of periods of this class of sequences and proving that in many cases 
the periods can be determined by the periods of their characteristic polynomials. Then we thoroughly 
investigate the periods of the polynomials in Zl(m){x] and show the existence, the construction methods, 
and the numbers of the polynomials with given periods. In particular, using these results, we conclude 
that there exist linear recurring sequences of order n over Zl(p‘) with maximum periods, which we call 
quasi-m- sequences, and their number is 

pP»-ix«-i )(1 -p-' fii e H.zA ' 


where p is a prime number, e £ 1 and p(-) is Euler's function. Finally, the number Nf(Z!(p‘)) of nonshift 
equivalent sequences generated by the same characteristic polynomial / (x) e Zl(p')\x} and the 
distribution function Z p e{b:u) of the number of occurrences of be Zl(p‘) in a full period of the linear 
recurring sequence u = (u,) are discussed. It is proved that if /(x)(mod p) is irreducible and the period of 
/(x)(mod p‘)isp r tc p (f), (0£r Se -1), then 
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It is further proved that for any b e Zl(p‘) 
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'"here K ? {f) is the period of f (x) (mod p ) m Zl(p)[x) R, is the period of /(x) in Zl(p‘)[x), S, is the 
period of u over Zl(p'), and S i is the penod of u (mod p ) over ZKp). By using the algorithm given in 
this paper, it is not difficult to determine K p (f ), R,, S,, and S i. Thus from these results the linear 
recurring sequences over Z/(m) are found to have many of the same beautiful properties as those over 
finite fields 
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SESSION ThA4 


SPEECH PROCESSING 


Fractional R ite Multi-Tree Speech Coding 

Jerry D. Gibson and Wen Whei Chang Dept, of Electrical Engineering, Texas A&M University, College 
Station, T-xos 77843 

We present both forward and backward adaptive speech coders that operate at 9.6, 12, and 16 
kbits/s using integer and fractional rate trees, unweighted and weighted squared error distortion measures, 
the (M, L) tree search algorithm, and incremental path map symbol release. We introduce the concept of 
multi-tree source codes and illustrate their advantage over classical, multiple symbol per branch, 
fractional rate trees for speech coding with deterministic code generators. With a frequency weighted 
distortion measure, the forward adaptive multi-tree coder produces near toll quality speech at 16 kbits/s, 
while the backward adaptive 9.6 kbits/s muld-tree coder substantially outperforms adaptive predictive 
coding and has an encoding delay less than 2 msec. Performance results are presented in terms of 
unweighted and weighted signal-to-noise ratio and segmental signal-to-noise ratio, sound spectrograms 
and subjective listening tests. 


Smoothed DPCM Codes 

Wen Whei Chang and Jerry D. Gibson Dept, of Electrical Engineering, Texas A&M University, College 
Station TX 77843 

Rate distortion theory promises that autoregresc.ve sources can be encoded optimally at small 
distortions (high rates) by a source coder with infinite encoding delay and zero delay at the decoder. 
However, for instrumcntable systems with finite encoding delay and an unmatched code generator or for 
operation at low rates, decoding delay may provide a performance increment. The alphabet constrained 
approach to data compression allows delay at both the encoder and the decoder, and Sethia and Anderson 
incorporate delay in a tree coder code generator by combining a weighted linear inteipolation scheme 
with DPCM. This system, called interpolativc DPCM (IDPCM), was shown to outperfoim DPCM at rate 

1 for several synthetic source models. In the present work, we use minimum mean squared error (MMSE) 
fixed-lag smoothing in conjunction with DPCM to develop a code generator employing delayed 
decoding. This smoothed DPCM (SDPCM) code generator is compared to DPCM and IDPCM code 
generators at rates 1 and 2 for tree coding several synthetic sources and to a DPCM code generator at rate 

2 for speech sources. The {M, L) algorithm is used for tree searching, and SDPCM outperforms IDPCM 
and DPCM at rale 2 for the synthetic sources with M = 1, 4, 8, and 12, and at rate 1 with M 24. For 
speech, SDPCM provides a slight improvement in MSE over DPCM codes that is also evident in sound 
spectrograms and subjective listening tests. The models upon which the fixed-lag smoother is based must 
be chosen appropriately to achieve good SDPCM performance. 


Deconvolution of Voiced Speech Based on Minimum-Phase/All-Pass Decomposition 
Ki Yong Lee, Iickho Song, and Souguil Ann Department of Electronics Engineering, Seoul National 
University, Seoul 151-742, Korea, and Department of Electrical Engineering, Korea Advanced Institute 
of Science and Technology, Seoul 130-650, Korea 

In this paper we view a speech signal as the convolved signal of the impulse response of a non¬ 
minimum phase vocal tract and a multi-pulse excitation source. A non-minimum phase can be 
decomposed into a minimum-phase and an all-pass component. Applying the linear predictive filter for 
the minimum-phase component and the spiking filter for the all-pass component, we propose a method 
for deconvolution of the muiti-pulse excitation source and the vocal tract The vocal tract is in the cascade 
form of a linear predictive filler and a spiking filter. Through computer simulation, we show the 
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efficiency of the proposed method using the convolved signal of a known non-minimum phase system 
and a multi-pulse excitation source and using teal speech. 


Modeling of Speech Excitation Source by a Bemoulli-Gaussian Process 

Ki Yong Lee, Iickho Song, and Souguil Ann Dept, of Electronics Engineering, Seoul National 
University, Seoul 151-742; Dept, of Electrical Engineering, Korea Advanced Institute of Science and 
Technology, Seoul 130-650; and Dept, of Electronics Engineering, Seoul National University, Seoul 
151-742, Korea 

In speech signal an excitation source consists of a few distinct spaise impulse train. In this paper the 
excitation source is modeled statistically as a zero mean Bemoulli-Gaussian process. The pulse locations 
are independently distributed with a probability distribution and pulse amplitudes are Gaussian random 
variables with zero mean and finite variance. In this paper we propose an algorithm using a likelihood 
function for estimation of pulse amplitudes and locations based on Bemoulli-Gaussian process. To obtain 
pulse location first we use the cross-correlation function between the speech signal and the vocal tract 
impulse response. Exploiting these pulse locations, and using a procedure in which the amplitudes of the 
pulses are kept optimum at each stage, we estimate optimum pulse amplitudes. Through computer 
simulation, we show that the proposed Bemoulli-Gaussian process model of excitation source is also a 
resonable model for the deconvolution procedure of real speech if we use maximum-likeiihood estimate. 


On the Use of Mean and Difference of Adjacent Line Spectrum Pair Frequencies for Speaker 
Recognition 

Chi-Shi Liu, Min-Tau Lin, Wem-Jyuhn Wang, and Jung-Juey Chen Basic Research Department, 
Telecommunication Laboratories, Ministry of Communication, PO Box 71, Chung-Li Taiwan, 320 
R.O.C 

The line spectram pair (LSP) was first introduced by Itakura as an alternative linear prediction (LP) 
representation in frequency domain. In the last years, a number of studies have shown that the LSP 
representation has more efficient encoding than other LP parametric representations and also provide 
better recognition rate than other spectral information in speech recognition. But still no papers discuss 
the role of LSP frequencies in speaker recognition and what performances they have. In this paper, we 
study the performance of LSP frequencies, and the mean and difference of adjacent LSP frequencies for 
speaker recognition. The result shows that LSP frequencies, the mean and difference of adjacent LSP 
frequencies could achieve 98%, 100%, and 100%, respectively by a codcbook with 16 codewords and test 
length with 7 digits. 


Predictive Coding for Stationary Gaussian Processes 

Kailash Birmiwal Department of Electrical Engineering, Southern Illinois University, Carbondale, 1L 
62901 

We consider predictive coding for stationary Gaussian processes and present a rate-distortion 
formulation of the coding problem. A simple novel iterative algonthm is developed for designing the 
entropy-encoded quantizer of the predictive encoder for all rates. The algonthm assumes Gaussian 
distnbution as the input to the quantizer, and the parameters of the distribution are updated at each stage. 
The performance of the algorithm is evaluated numencally, and the results are compared with the 
optimum distortion-rate functions. 


-IV 
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A Space-Variant Covariance Model for DC/AC-Separated Image Block Coding 

Yonggang Du Institut fur Elektrische Nachrichtentechnik, Aachen University of Technology, D-5100 

Aachen, West Germany 

Forblockwise image source ceding each picture is partitioned into rectangular blocks of equal size. 
The statistics, especially the covariance matrix of the blocks, have widely been assumed to be stationary 
within the block field. In this paper we will show that this assumption is only conditionally correct. It is 
tme if the blocks are processed along with their DC-value. In many practical applications, however, the 
mean value (DC-value) of each block is often removed from the block and handled in its own manner, 
because of its weak cotreladons to the AC-values. One has in fact a DC/AC-separated block coding. In 
this case the actual Mocks to be modeled are those whose DC-term is removed. For such mean-removed 
blocks it is now shown that their covariance matrix is no longer space-invariant A mathematical proof is 
presented in this contribution, confirmed by experimental measures. Model functions are derived for 
variable block size and for blocks with or without classification in orientation angles. 


Gain Adapted Hidden Markov Models for Recognition of Clean and Noisy Speech 
Yariv Ephraim Speech Research Department, AT&T Bell Laboratories, Murray Hill, NJ 07974 

A key issue in applying hidden Markov modeling for recognition of speech signals is the matching 
of the energy contour of the signal to the energy contour assumed by the model. A mismatch between the 
energy contours could cause a major problem for speech recognition, since the probability of speech 
events could be miscalculated by the model. When clean signals are available for recognition, the gain 
matching is usually achieved by appropriately normalizing the signal prior to both training and 
recogniuon. When only noisy speech signals are available for recognition, the standard approach of gain 
normalization is not applicable. In this paper, a unified approach is developed for designing models for 
gain normalized signals, and for gain adaptanon in recognition of clean and noisy speech signals. During 
training, the gain function of the speech signal is estimated along with the parameter set of the model. 
During recognition, joint gain estimation and word decoding is perfoimed. The gain function and 
parameter set are exclusively estimated by the maximum likelihood estimation approach using the EM 
(estimation-maximization) algonthm. 
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SESSION ThA5 


BLOCK CODING 


Constructions of Error-Correcting DC-Block Codes 

Tuvi Etzion Computer Science Dept., Techmon, Haifa 32000, Israel 

A (ln,l,c,d) dc-free binary block code is a code of length 2n, constant weight n, maximum 
mnlength of a symbol I, maximum accumulated charge c, and minimum distance d. The requirements are 
that / and c be small. We present two dc-free codes with distance 2d, dt\, length 2m+2r(d-\) ford£3 
and length 2m +2r(d-l)(2d-l) ford >3, where rS[log 2 (2m+ 1)]. For the first code /=4, c=2, and the 
asymptotic rate of this code is 0.7925. For the second code 1 =6, c =3, and the asymptotic rate of this 
code is 0.8858. Asymptotically these rate achieve the channel capacity. For small values of n these codes 
do not achieve the best rate. As an example for codes of short length with a "good" rate, we first present a 
(30,10, 6,4) dc-free block code with 2 2 ' codewords. Finally, we present a construction for which from a 
given code C i of length n, even weight, and distance 4, we obtain a (4n, l, c, 4) dc-free block code C 2 , 
where i is 4,5, or 6, and c is not greater than n +1, but usually significantly smaller. The codes obtained 
by this method have good rates for small lengths. We discuss the encoding and decoding procedures for 
all the codes. 


A Class of Error and Erasure Control (d.k) Block Codes 

H. C. Ferreira and Shu Lin Laboratory for Cybernetics, Rand Afrikaans University, P.O. Box 524,2000 
Johannesburg, South Africa, and Department of Electrical Engineering, University of Hawaii at Manoa, 
2540 Dole Street. Honolulu, HI 96822 

We show that a unique integer composition can be associated with each (d.k) sequence. By 
imposing some restrictions on such a composition, block codes capable of error and erasure control can 
be synthesized. We first present three families of restricted (d,k) sequences capable of single error 
detecuon. If we append a small and fixed number of parity and buffer bits to these rcstneted (d.k) 
sequences of arbitrary length, codes capable of correcting single errors and up to (d +1) adjacent erasures, 
can be constructed. Codes capable of correcting all single and double adjacent errors and codes capable of 
correcting up to (2d + 2) adjacent erasures, can be constructed either by imposing further compositional 
restnetions or by increasing the number of panty bits. For each composiuon, we present difference 
equations for enumeration, characteristic polynomials and tables with numerical capacities. The rates of 
the class of codes investigated here, approach the capacity of the d constrained channel for large d. In 
companson to recording codes currently used, some of our constructions exceed the minimum separation 
between transitions in the channel waveform, at the expense of a smaller detection window width. 


On Error-Controlling (d.k) Constrained Block Codes 

^tvind Ytrehus Dept, of Informatics, Thormthlengst. 55, N-5006 Bergen, Norway 

Binary (d.k) constrained, possibly dc-free, error detccung or correcting block codes are considered 
with regard to three types of channel errors: binary-symmetric errors, asymmetric errors and biishift 
errors. Bounds on the code sizes are obtained from computer search techniques, constructions and 
sphere-packing bounds. Short block codes can be used as building blocks in longer concatenated (and 
generalized concatenated) code constructions. 
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On Multi-level Block Modulation Codes 

Tadao Kasami, Toyoo Talcata, Torn Fujiwara, and Shu Lin Faculty of Engineering Science, Osaka 
University, Toyonaka, Osaka 560, Japan, and Department of Electrical Engineering, University of 
Hawaii at Manoa, 2540 Dole Street, Honolulu, HI 96822 

One of the dramatic developments in bandwidth-efficient communications over the past few years is 
the introduction and rapid application of combined coding and bandwidth-efficient modulation, known as 
coded modulation. Using coded modulation, reliable data transmission can be achieved without 
compromising bandwidth efficiency. In this paper, we investigate the powerful multi-level technique for 
combining Mock coding and modulation. This multi-level technique allows us to construct bandwidth- 
efficient block modulation codes with aibitrary large minimum squared Euclidean distances from 
Hamming distance component codes (binary or nonbinary) in conjunction with proper signal mapping. 
The paper consists of four parts. In the first pan, we present a formulation for signal sets on which 
modulation codes are to be constructed. Distance measures on a signal set are defined and their properties 
are developed. In the second pan, we present a general formulation for multi-level modulation codes in 
terms of component codes with appropriate Euclidean distances. The distance properties, Euclidean 
weight distribution and linear structure of multi-level modulation codes are investigated. In the third part, 
several specific methods for constructing multi-level modulation codes are proposed. Based on these 
methods, some short block codes for 8-PSK, 16-PSK and 16-QASK modulations are constructed. These 
codes have good minimum squared Euclidean distances and provide significant coding gains over some 
uncoded reference modulation systems with little or no bandwidth expansion. In the last pan, enor 
performance of block modulation codes is analyzed for an AWGN channel based on a soft-decision 
maximum likelihood decoding. Error probabilities of some specific codes are evaluated based on their 
Euclidean weight distributions. 


On Linear Structure and Phase Rotation Invariant Properties of Block 2 ( -ary PSK Modulation 
Codes 

Tadao Kasami, Toyoo Takata, Torn Fujiwara, and Shu Lm Faculty of Engineering Science, Osaka 
University, Toyonaka, Osaka 560, Japan, and Department of Electrical Engineering, University of 
Hawaii at Manoa, 2540 Dole Street, Honolulu. HI 96822 

As the application of coded modulation in bandwidth-efficient communications grows, there is a 
need of better understanding of the structural properties of modulation codes, especially those properties 
which are useful in: error performance analysis, implementation of optimum (or suboptimum) decoders, 
efficient resolution of carrier-phase ambiguity, and construction of better codes In this paper, we 
investigate two important structural properties of block 2'-ary PSK modulation codes, namely, linear 
structure and phase symmetry. For an AWGN channel, the error performance of a modulation code 
depends on its squared Euclidean distance distnbution. Linear structure of a code makes the error 
performance analysis much easier. Phase symmetry of a code is important in resolving carrier-phase 
ambiguity and ensuring rapid carrier-phase resynchronization after temporary loss of synchronization. It 
is desirable for a modulation code to have as many phase symmetries as possible. In this paper, we first 
represent a 2 , -ary modulation code as a code with symbols from the integer group, G = (0, 1,..., 2 ( -l), 
under the modulo^ addition. Then we define the linear structure of a block 2'-ary PSK modulation code 
over G with respect to the modulo-2' vector addition, and derive conditions under which a block 2'-ary 
PSK modulation code is linear. Once the linear structure is developed, we study phase symmetry of a 
block 2 ( -ary PSK modulation code. In particular, we derive a necessary and sufficient condition for a 
block 2 , -ary PSK modulation code, which is linear as a binary code, to be invariant under 180°/2 f '* phase 
rotation, for 1 £ h Sf. Finally, a list of short 8-PSK and 16-PSK modulation codes is given together with 
their linear structure and the smallest phase rotation for which a code is invanant. 
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A Class of Block Codes with Redundant Signal-Sets for PSK-Modulation 

Magnus Jsaksson and Lars H. Zetterberg Telecommunication Theory, Royal Institute of Technology, S- 
100 44 Stockholm, Sweden 

This paper deals with channel codes where the redundancy is obtained not from parity symbols, but 
from expanding the channel signal-set. They were initially proposed by Ungerboeck using a 
convolutional code. In this paper, we give a block coding approach. 

The expanded signal-set is given the structure of a finite field. The code is defined by a square 
nonsingular ciroulant generator matrix over the field. Binary data is mapped on a dataword, of the same 
length as the codewords, over an additive sub-group of the field. A computer search has resulted in, e.g., 
rate 2/3 coded 8-PSK of lengths 4-8 with coding gains 0.7-3.0 dB over uncoded QPSK. 

We describe the codes using trellises, and then apply the Vtterbi algorithm for decoding. For 8- 
PSK, there are codes with gain 3 dB with only 4 states in the trellis. 

We also show that parallel transitions in the trellis can be described by an additive group, specified 
by the im erse generator matrix, and its cosets. 


More on the Behavior of Binary Block Codes at Low Signal-to-Noise Ratios 
Chi-chao Chao and Robert J. McEliece Department of Electrical Engineering, National Tsing Hua 
University, Hsinchu, Taiwan 30043, R.O.C., and Department of Electrical Engineering, California 
Institute of Technology, Pasadena, CA 91125 

It is well-known that for block codes of a given rate, the larger the minimum distance, the better the 
code will pcrfoim at high signal-to-noise ratios. For some applications such as deep space 
communications, we care about the behavior of codes at low signal-to-noisc ratios. In a foimcr paper by 
Swanson, McEliece, and Chao, an expressions was derived for the block error probability of binary block 
codes on the ui.quantizcd additive white Gaussian noise (AWGN) channel near the point where the 
signal-to-noise rauo is zero. In this paper we find a similar expression for the bit error probability with 
maximum-likelihood decoding. Examples of codes such as orthogonal codes, bi-orthogonal codes, the 
(24,12) extended Golary code, and the (15,6) expurgated BCH code are discussed. The asymptotic coding 
gain on the unquantizcd AWGN channel at low signal-to-noise ratios is also studied. (This work was 
supported by the Air Force Office of Scientific Research under grant # AFOSR-88-0247.) 


Block Coded Modulation on AWGN and Fading Channels 

Lin Zhang and Branka Vucetic School of Electrical Engineering, University of Sydney, NSW 2006, 
Australia 

In this paper, new BCM codes for 8-PSK modulation with good error rate performance are 
presented. The performance of these codes on the addiuve white Gaussian channel (AWGN) and the 
Rayleigh fading channel is evaluated. 

A fading channel, with its specific properties, requires different entena in constructing short BCM 
codes from those used for an AWGN channel. Although minimum squared Euclidean distance (dp,,) is 
still very important, it plays a less significant role when dealing with severe facing channels Instead, 
effective length (f,^) along the path is considered for code construction. Our results show that the 8-statc 
BCM code without parallel transitions in its trellis diagram is supenorto those with parallel transitions 
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SESSION ThA6 


CODING THEORY VI 


A Truncated-Stack Sequential Decoding Algorithm: Analysis and Implementation 

Pierre Lavoie, David Haccoun, and Yvon Savaria Department of Electrical Engineering, Ecole 

Polytechnique de Montreal, P.O. Box 6079, station “A", Momre'al (Quebec) Canada 

A new sequential decoding algorithm called Truncated-Stack Algorithm" (TSA) is presented and 
shown to be attractive for high speed decoding of long convolutional codes. In this algorithm, a 
significant reduction in stack size is achieved by gradually discarding from the stack some paths that are 
very unlikely to be correct. Computer simulations show that the cumulative distribution of the 
computational effort is the same for the TSA with stack size S and the usual stack algorithm as long as the 
number of computations remains smaller than a critical value C cra (S). An analysis based on the theory of 
branching random processes confirms that S « C cru (S), implying that the TSA can be more economical 
to implement than die usual stack algorithm. 

The problem of implementing TSA decoders for high-speed applications is examined. A multi¬ 
processor VLSI architecture based on a new systolic prionty queue is presented. The new systolic pnority 
queue allows a very fast ordering of the nodes in the stack, hence making TSA attractive for applications 
requiring powerful error correction at very high data rates. 


Sequential Decoding Without a Cut-off Rate 

J. B. Anderson. ECSE Department, Rensselaer Poly Inst., Troy, NY 12180-3590 

Traditional sequential decoders are designed from the premise that correct decoding of a data 
symbol must occur with certainty. A standard analysis then calculates the expected number of code tree 
branches that are visited in the decoding of each symbol Some difficulties anse in this kind of design 
The branch expectation is infinite at code rates above R 0 , the computational cut-off rate. The decoder will 
violate any hard limit to storage or computing power with at least some probability, even at rates below 
R 0 \ it then fails to decode correctly, in conflict with the premise. We investigate decoders that arc 
designed from the premise that they may fail to decode correctly with probability P, > 0. Suppose that all 
code tree paths arc traced on a plot of distance to the received path vs. Path length. It turns out that a P,- 
dccoder must explore all the paths between two boundaries, a drop line and a stop line. Paths may be 
dropped that hit the drop line, and any path hitting the stop fine stops the search; its initial symbol is the 
decoder output. We give simple expressions for these two lines and for the expected branches visited. 
These are given for the optimal code searches of two types, those that allow backtracking in the search 
and those that do not. The channel is the binary symmetric channel with crossover p. Neither type of 
search shows a cut-off rate phenomenon; the expected branches visited simply grows to infinity as p tends 
to p c , the crossover probability leading to a capacity equal to R. The expectation is of the form 
exp[[l + (log o)l h] log P,\ for searches that backtrack and exp[[(logcr)/A JlogP,] for those that do not 
Here h and a are functions of p and R. Furthermore, P, and the maximum depth of search n arc related by 
P, = exp[-«£■(/?)], where E(R) is the Gallager reliability exponent for block codes. The behavior of 
these P,-decoders can be directly related to existing decoders such as the stack, Fano, M-, and Viterbi 
algorithms. 
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Sequential Decoding and Wald’s Identity 

Rolf Johannesson and Kamil Sh. Zigangirov Department of Information Theory, University of Lund, Box 
118, S-221 00 Lund, Sweden, and Institute for Problems of Information Transmission, USSR Academy of 
Sciences, 19 Ermolovoy st., Moscow GSP-4, USSR 101447 

In this paper we give a unified analysis of sequential decoding. All results are based on Wald’s 
identity. First we derive an upper bound on the average number of computations per branch for the stack 
algorithm. A similar bound is shown for the Fano algorithm. For finite random tree codes terminated by 
tails of dummy information zeros we give an upper bound on the error probability which is valid for both 
algorithms. For infinite trees we introduce a fixed backsearch limit and derive an upper bound on the error 
probability which holds for the ensemble of fixed convolutional codes if the memory of the encoder 
m = n -1, where n is the backsearch limit. Finally, the different choices of the bias term in the metric are 
discussed. 


Orphans of the First Order Reed-Muller Codes 

Richard A. Brualdi and Vera S. Pless Dept, of Mathematics, University of Wisconsin, Madison, 
Wisconsin 53706, and Dept of Mathematics, University of Illinois at Chicago, Chicago, Illinois, 60680 
If C is a code, an orphan is a coset which is not a descendant. Orphans arise naturally in the 
investigation of the covering radius. In case C has only even weight vectors and minimum distance at 
least 4, we characterize cosets which are oiphans, and then prove the existence of a family of orphans of 
first-order Reed-Muller codes R(l,m). Form £5 all orphans off? (l.m) are identified. 


Coset Codes with Isometric Labelings 

G. David Forney, Jr. Codex Corporation, 20 Cabot Boulevard, Mansfield, MA 02048 

A coset code C (A/A'; C) based on a lattice partition A/A' results from mapping the sequences y in 
some group code C to sequences of cosets of A' via a labeling function that maps code outputs (labels) y* 
to coscts of A'. If the labeling function is linear, then 4 is linear. Isometric labelings are a broader class 
that lead to codes (’pseudolinear codes’) that have geometrical invariance properties similar to those of 
linear codes. In particular, a pseudolinear code is distance-invariant, and all of its Voronoi regions have 
the same shape. These properties extend to ‘label translates’ of pseudolinear codes. All known good 
lattice-type cosct codes arc pseudolinear. Linear labelings, or translates of linear labelings, are isometric, 
and isometnc labelings arc regular. A binary isometric labeling exists if andonlyif Z N IAIA'I4Z N is 
a lattice partition chain, whereas a linear labeling exists if and only if Z^/A/A72Z" is a lattice partition 
cham (for some scaling of A/A'). Furthermore, if there is an isometnc labeling for A/A', then there is an 
isometnc Ungcrboeck labeling for A/A'. If 4 is a trellis code based on a linear time-invanant 
convolutional code C, then the time-zero lattice of 4 is a pseudolinear code Ao, and the set of possible 
outputs from any encoder state is a label translate of Ao. 


Operating Cosets in Arbitrary Lattice Partitions 

Mauro A. 0. da Costa c Silva, Faculdade de Engenhana Elelnca, Untversidade Estadual de Campinas, 
Caixa Postal 6101, Campinas-SP, Brasil 

In the computational search for optimum trellis coded modulation (TCM) schemes based on lattice 
partitioning, as proposed b Caidcrbank and Sloane, it is convenient to generate an addition table for a set 
of cosct representatives, each time a different partition 17A of a lattice P by a sublattice Ac T is used, in 
order to simplify the search for good generalized convolutional codes based on T/A, whose output cosct 
is a linear combination of some cosets in TIA chosen as the columns of its generator matrix. A special 
selection of coset representatives for arbitrary lattice partitions is proposed and a closed analytical 
expression for addition of these cosct representatives is derived. Besides ns application in the search for 
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optimum TCM schemes, the material could be used to verify whether a given periodic packing is indeed a 
lattice. 


New Lower Bounds for Asymmetric Codes 

Tuvi Etzion Computer Science Department, Technion - Israel Inst, of Technology, Haifa 32000, Israel 
We present three different methods to obtain lower bounds for binary asymmetric codes. The first 
method is by using some combinations of the Cartesian product between a partition of all the binary n- 
tuples into disjoint constant weight codes with distance 4 and a partition of all the binary k-tuples into 
disjoint asymmetric codes with distance 2. By using this method we obtain lower bounds for binary 
error-correcting codes of length n + k with asymmetric distance 2. The second method is by combining 
the Preparata code of length 2* (or the shortened Preparata code of length 2“ -1), the extended Hamming 
code (or the Hamming code), and some binary i-tuples. By using this method we obtain codes of length 
2" + * (or 2" + *-l) and asymmetric distance 3. the third method is by considering the weight 
distribution of known codes and their translates. In a number of cases the results significantly improve on 
the best lower bounds previously known. 


Sequential Decoding with an Incremental Redundancy ARQ Scheme 

S. Kallel Department of Electrical Engineering, University of British Columbia, 2356 Main Mall, 
Vancouver, B C. Canada V6T1W5 

In this paper, sequential decoding is analyzed in conjunction with an efficient incremental 
redundancy ARQ scheme using punctured convolutional coding. With the incremental redundancy ARQ 
scheme, whenever the decoding ume for a given data packet exceeds some predetermined value T ma , 
decoding of that packet is stopped and incremental redundancy bits arc provided by the transmitter, 
decreasing thus the coding rate. Should decoding still fail, then the transmitter sends additional 
incremental redundancy bits, decreasing once again the coding rate. This procedure continues until 
decoding finally succeeds. 

It is shown that with sequential decoding used in conjunction with the incremental redundancy 
ARQ scheme, the throughput increases as the starting coding rate increases. Moreover, it is shown that 
with a starting high coding rate code, the throughput is always better than with a rate 1/2 code with code 
combining. Thus, the incremental redundancy ARQ scheme makes sequential decoding very powerful, 
allowing the communication system to be flexible and adapuve to channel conditions, even under wide 
noise vanations and severe channel degradations 


129 


SESSION ThA7 


VITERBI DECODERS 


Windows, Multipath and Viterbi Modems 

Torieiv Maseng and Odd Trandem Elab-Rumt, N-7034 Trondheim, Norway 

A Viterbi decoder is used to demodulate a signal conupted by noise and multipath. A new 
technique is presented for choosing the part of the channel impulse response which minimizes the 
intersymbol interference caused by a lengthy channel response, considering the finite complexity of the 
receiver. This enables longer multipath profiles to be equalized without increasing the number of states. It 
is efficient when applied to nonlinear modulation techniques with compact power spectra. Such nonlinear 
modulation schemes are of interest because they may contain inherent coding gain. These schemes are 
exemplified. 


Error Computation of Viterbi Decoder 

H. F. Rashvand Technophone Ltd., Research Dept., Surrey, GUIS 3SP, England 

A new model for calculating the output error probability of a convolutional code that uses the 
Viterbi Decoding Algorithm (VAD) is introduced. Its structure is based upon the Markov chains model of 
the system. It makes use of a state transition matrix and an error generating vector. Then from this model 
two programmable algorithms are derived corresponding to two different implementauons of the VAD In 
the first case the error generating vector is initially set at the error free condition and then taken through 
the trellis while adjusted step-by-step by applying a transition matrix. The vector follows a chain of 
identical successive processes that eventually reaches a final state for the specified search length. The 
error computation of the code then becomes a straightforward accumulation of an error contributiig 
subset of the vector. For the second case, however, first a state-metric transition matrix is generated T.iis 
matrix is then converted into an error generating vector that is adjusted according to decoder’s search 
length. The rest of this algorithm is similar to the first case. The accuracy of the algonthms is compared 
with results of the Monte Carlo Method and illustrated against some upper and lower bounds. 


A Burst Error Model of Viterbi Decoding for BPSK Modulation on Fading and Scintillating 
Channels 

Joel M. Morris and Dcval Patel Electrical Engineering Department, University of Maryland Baltimore 
County, Baltimore, MD 21228 

The burst error statistics of a soft-decision Viterbi decoder were simulated and a corresponding 
model obtained for the case when the Lansmittcd signal is: (I) encoded with the 313 (3, 1/2), 31123 (5, 
1/2), or 3233013 (7,1/2) convolutional codes; (2) modulated via coherent BPSK for the AWGN channel; 
and (3) subjected to slow and non-selective scinlillation/fading modeled by the Nakagami-m distnbutioa 
These statistics were generated by Monte-Carlo simulations, and compiled in terms of burst-error-Icngth 
average and quantile (.90%, and 99%) statistics vs SNR (E t /N a ) parameterized by the fading intensity 
parameter m. These statistics are evaluated to detenu ine the appropnateness of extending the geometric 
burst-error model for the nonfading case to the case of Nakagami-m fading. The results have important 
implications tor the design of interleaved or non-intcrlcavcd concatenated coding schemes, and to the 
development of burst error models and simulators for scimillatmg and fading channel environments 
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Generalized Viterbi Algorithms (GVA) for the Decoding of Convolutional Codes 
Nambirajan Seshadri and Cari-Erik W. Sundberg AT&T Bell Laboratories, Murray Hill, NJ 07974 

Two generalizations of the Viteibi algorithm (VA) are presented for the decoding of convolutional 
codes. They are respectively, a parallel version dial simultaneously identifies the L globally best 
candidates (L > 1), and a serial version that iteratively finds the L best candidates. An erasure is declared 
if the metric difference of the best and the second best path falls below a certain threshold. Such an 
infonnation is useful in combined source and channel coding, where an unreliable frame of decoded data 
can be replaced using inter-frame source redundancy either through the process of prediction or 
interpolation. Alternatively, the inter-frame source redundancy can be used to select the best out of the L 
candidates that are released by the channel decoder. In data transmission schemes, an outer encoder adds 
intra-frame data redundancy to the data to be encoded by the inner convolutional code. At the receiver, 
the inner decoder, which is based on the GVA, produces a list of the L best candidates, and the outer 
decoder selects that candidate that satisfies the intra-frame data redundancy check. Gains of more than 1.0 
dB are possible using GVA with L - 3 over the VA(L = 1) on the AWGN channel. The modulation is 
BPSK and the inner code is a rate R = 1/2, memory M - 4 convolutional code. For the Rayleigh fading 
channel, a gain of more than 2 dB is obtained. Signal space geometry ideas are used to illustrate how 
these gains are achieved. For the AWGN channel, a lower bound 0 ” the asymptotic coding gain for 
different values of L are presented for the convolutional codes by using the simplex signal set. 


Multistage Decoding Using a Soft-Output Viterbi Algorithm 

Joachim Hagenaucr and Peter Hoeher German Aerospace Research Establishment (DLR), Institute of 
Communications Technology, D-8031 Oberpfaffcnhofcn, West-Germany 

The Viterbi algorithm (VA) is modified to deliver not only the most likely path sequence in a 
finite-state Markov chain, but either a probability estimate or an analog (‘soft’) value for each symbol. 
This reliability indicator allows for making soft decisions used for decoding outer codes. In such a way 
the inner VA accepts and delivers soft sample values and can be regarded as a device for improving the 
SNR, similar to an FM demodulator. Several applications of multistage decoding are investigated to show 
the gain over the conventional hard-deciding VA including concatenated convolutional codes. The Soft- 
Output Viteibi Algorithm (SOVA) can also be applied to the concatenation of convolutional and simple 
block codes, the concatenation of trellis-coded modulation and convolutional FEC codes, and to coded 
Viterbi equalization. For these applications we found additional gains of 1 to 4 dB as compared to the 
classical hard-deciding algorithms. 


Quantization Effects in Viterbi Decoding 

Ivan M. Onyszchuk, Kar-Ming Cheung, and Oliver Collins California Institute of Technology and Jet 
Propulsion Laboratory, Pasadena, CA 91125 

In order to design efficient Viterbi decoders, the tradeoff between the number of input quantization 
bits 1 1 and performance loss must be known. An optimal (uniform) quantization method for the AWGN 
channel is presented. The corresponding channel capacity C u (q) is used to determine the smallest value of 
q that does not substantially increase a q = ~ decoder’s bit error rale (BER). The quantizer stepsize which 
maximizes C M (q) almost minimizes the decoder BER. The range of state metrics is analyzed to determine 
a small state metnc register width. We describe a simple renormalization scheme that will be used in 
JPL's new K = 15, rate 1/6, multichip decoder for the Galileo deep-space missioa These results yield 
design parameters for reduced hardware complexity Viterbi decoders with negligible BER increase from 
maximum likelihood decoding on the AWGN channel. 
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High Speed Viterbi Decodings Having an Idle Mode 

Kazuhiko Yamaguchi and Hideki Imai Department of Computer Science and Information Mathematics, 
University of Electro-Communication and Division of Electrical and Computer Engineering, Yokohama 
National University, Yokohama, Japan 

This paper is concerned with high speed Viterbi decoding algorithm having two modes (i.e., 'active 
mode’ and ‘idle mode'). By the evaluation of the algorithm, we show a new criterion of good 
convolutional codes for the algorithm. The basic concept of the algorithm is that the Viterbi decoding is 
performed only when errors are detected. Even if we use a component of the slow-speed Viterbi decoder, 
the algorithm can attain high communication speed. For example, the case of rate 1/2 convolutional code 
with 6 encoder memories realize 100 times higher communication speed than that of the component 
Viterbi decoder at the point of channel bit error rate 1(T 3 . By using the transfer functions, we prove the 
algorithm is guaranteed to be maximum-likelihood decoding (MLD) for all noncatastrophic convolutional 
codes. From the evaluation of the condition to be MLD for each convolutional code, we give the suitable 
generator polynomials of convolution code for the algonthm. The analysis method is related to the study 
of the error probability of Viterbi decoding with truncated memory. The interesting result is given in the 
presentation. 


High Speed Viterbi Decoder Structures 

Enk Paaske Institute of Circuit Theory and Telecommunications, 343, Technical University of Denmark, 
DK-2800 Lyngby, Denmark 

We propose new Viterbi decoder structures which exploits the more powerful unit memory or 
parual unit memory codes, decodes more than one bit per clock cycle and have only few operations per 
decoded bit. They are suitable for implementation with advanced standard integrated circuits, in 
particular large memory chips. Data rates from 150 Mbit/s to more than 300 Mbit/s ware achievable 
without using multiplexed decoders. 

The structure is based on the state space of the syndrome former The state metrics and decisions in 
level f +1 can be expressed by 

Mw =/(M, s Sf,i) and D w =g(M ri /,Sy f |) 

where M| denote State metrics at level f and S|+i the syndrome generated by the received bits. By 
appropriate renormalizing of M, the number of metnc combinations N Q is final, and if it is small enough, 
the functions in (1) can be performed by a single ROM which replaces all ACS circuits in a conventional 
decoder For hard decisions powerful codes directly suitable for such structures are found and for soft 
decisions, where N c are generally quite large, methods are developed to reduce N c in such way that no or 
only a small performance degradation results. 


132 


TECHNICAL SESSIONS 
Thursday, 2 p.m. - 5 p.m. 


SESSION ThPl 
DETECTION THEORY H 


Fast Simulation of Detector Error Probabilities in the Presence of Memory and Non-Linearity 
Randall K. Bahr and James A. Bucklew Department of Electrical & Computer Engineering, University 
of Arizona, Tucson, AZ 85721, and Department of Electrical A Computer Engineering, University of 
Wisconsin-Madison, Madison, Wl 53706 

One would like to compare and analyze digital communication systems based upon their overall 
probability of error. Unfortunately, useful closed form expressions for these probabilities arc almost 
impossible to derive due to the complexity of the stochastic system. Hence one must resort to simulation 
techniques. One such technique is Monte Carlo simulation, which directly counts the number of errors in 
repeated trials. The error probabilities are usually quite small, requiring numerous simulation runs to 
sufficiently "hit" the rare event, a bit or symbol error, to gain adequate knowledge of its statistics. This 
places severe demands on the computer's random number generator. Importance sampling strategies 
simulate under altered input signal distribuuons (translation or stretching) so as to "speedup" convergence 
of the error estimators. In this paper wc discuss a speedup technique based upon the observation that the 
error esumator’s variance vanishes exponentially fast as sample size increases. The simulation is earned 
out under a new input signal distribution which is adjusted to maximize the rate of exponential decrease. 
The fast simulation method is compared to two other importance sampling techniques currently in use 


Distributed Detection with Decision Feedback 

Rajan Srinivasan Department of Electrical and Computer Engineering, Syracuse University, Syracuse, 
NY 13244-1240 

A distributed detection scheme using feedback is proposed and studied. On each observation, 
peripheral detector decisions arc combined in a central processor. Feedback links from center to each 
peripheral exist. Using a simple traffic-reduction protocol, the central decision is communicated to all 
peripheral detectors and serves to update peripheral decisions on a new observation. The sequence of 
hypothesis tests representing the decision process is shown to be consistent. It is argued that overall link 
usage can be appreciably reduced for this scheme. Performance results for a Rayleigh channel arc 
presented and suggestions for further investigations made. 

Optimum Detection in the Presence of Random Transient Disturbance and White Gaussian Noise 
T T. Kadota AT AT Bell laboratories, Murray Hill, NJ 07974 

Dctecuon of a deterministic signal in white Gaussian noise and a random transient disturbance is 
considered. Instead of the usual likelihood ratio as the detection stalisuc, we propose the use of the 
conditional likelihood ratio given the disturbance where the disturbance is estimated by the maximum a 
posteriori likelihood method. Since the likelihood ratio is the conditional version averaged over all 
possible realizations of the disturbance but the disturbance in the data, if any, is one particular realization, 
use of the conditional version is more appropnatc. The disturbance model chosen is a piece of Gaussian 
process multiplied by a Rayleigh-distributed constant vhcre the starting time and the duration are 
uniformly distributed. Such a distuibancc, together with white noise, constitutes the worst case of the 
contaminated Gaussian noise in the min-max robust detection The detection procedure in the presence of 
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this disturbance consists of two steps: (i) first to determine whether or not the disturbance is present and, 
if it is present, (ii) then to estimate it and subtract the estimate from the data before the matched-filtering. 
These two steps tend to be computation-intensive and various ad hoc but more efficient schemes are being 
investigated. 


The Use of ARE in Finite Sample Size Detector Performance Prediction 

Rick Blum and Saleem A. Kassam General Electric Co., and Department of Electrical Engineering, 
University of Pennsylvania, Philadelphia, PA 19104 

The limitations of the asymptotic relative efficiency (ARE) in predicting finite sample size detector 
performance have been noted in several previous studies. It has been observed that the finite sample size 
relative efficiency (RE) will often converge very slowly to its asymptotic limit In addition, in some cases 
the RE approaches the ARE from below while in other cases the RE overshoots the ARE and approaches 
it from above. It is demonstrated in this paper that both of these effects can be predicted for a large and 
useful class of detectors used for detection of known signals in additive noise. The prediction formula is 
developed through analysis for a general detector structure. The result is then used to analyze some 
specific detectors. Among the specific detectors are the sign detector, the dead-zone detector, the four- 
level detector, and the more general iV-level quantized detector. In addition, the soft-limiter and the noise 
blanker are analyzed. Computer calculations and simulations giving the relative efficiency as a function 
of sample size have been used to verify our predictions. These results are also presented. 


Reduced State Sequence Detection of Continuous Phase Modulation 

Ame Svensson Ericsson Radar Electronics AB, Aerospace Division, Airborne Radar Systems Design, S- 
431 84 Molndal, Sweden 

A reduced state sequence detector (RSSD) which combines decision feedback or hard decision with 
Vucrbi decoding is proposed for Continuous Phase Modulauon. With decision feedback, this detector can 
successfully be used for both binary and high-level CPM schemes. This detector is analyzed by means of 
minimum Euclidean distance and by simulations of the symbol error probability. 

It is shown that many CPM schemes can be decoded without using phase states in the decoder 
without sacrificing asymptotic error performance (minimum distance). For some schemes, especially 
high-level schemes, the number of stales can be further reduced while still maintaining asymptotically 
optimum error performance Furthermore, a very small number of slates can be employed in the decoder, 
with a relatively small degradauon in the error performance. With hard decision, a detector not making 
use of phase states can be used for small modulauon indices UP with a minor degradation in error 
performance. 


Quickest Detection of Time-Varying Signals 

S. D. Blostcin Department of Electrical Engineering, Queen’s University, Kingston, Ontario K7L 3N6, 
Canada 

In the past, research in quickest-dctection has been confined to the detection of a change from one 
stauonary stochastic process to another. In many situations, signals of interest may be time-varying, as in 
the problem in delecting slowly emerging targets in radar clutter or in rapidly detecting sampled pcnodic 
signals such as for earner synchronization in coherent communications systems. A quickest-dctection 
algorithm for time-varying changes in the means of a discretized data sequence is presented in this paper. 
This is accomplished by generalizing reaction procedures studied by Page and Lorden to the time-varying 
case. Bounds on false alarm rate are derived using a generalized version of a theorem by Lorden and 
applied to the test design problem of threshold selection. In addition, an upper bound of the average 
reaction time of the quickest u tion algorithm is derived for the case of white Gaussian noise. To test 
the theory, the analytical results are compared to simulations performed on discretized step-functions, 
ramps, and penodic functions of known frequency in white Gaussian noise. 
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Threshold Signal Detection and Estimation in Signal-Dependent Noise 
David Middleton 127 East 91 Street, New York, NY 10128 

Recent analyses of threshold signal detection (D) and estimation (E) involving generalized noise 
and interference have dealt primarily with "telecommunication” environment' where the significant noise 
is ambient, i.e., does not depend on the original transmitted signal. However, for many applications 
additional scattered or (unresolvaNe) signal-dependent components occur, often accompanied by 
(resolvable) multipath. This feature, of course, has been extensively considered in earlier work, but under 
gauss noise conditions and usually without explicit physical structure. But for the many instances of 
small-angle, high-frequency operation, and where only a few significant scatterers are involved, among 
other conditions, this noise is no longer normal, and can be highly nongaussian. 

Accordingly, it is necessary to extend the earlier canonical theory. The present paper accomplishes 
this by appropriately treating the scatter component as an additional signal. This also lowers the model’s 
statistical burden, since only second-order second-moments (covariances), at most, of these new 
"signals”, are required. (The first-order pdf of the ambient noise, as before, is still needed in these 
independent sampling cases.) The resulting threshold algorithms and performance measures arc 
noticeably modified. Performance may be enhanced or degraded, depending on the statistics of the 
scattering model. This is illustrated by examples, including radar and sonar, as well as telecommunication 
applications. 


Weak Signal Detection in Correlated Non-Gaussian Noise 
David Middleton 127 East 91 Street. New York, NY 10128 

Previous work of the author and others in developing an effective canonical theory of weak-signal 
detection in nongaussian noise has required the assumption of independent sampling. While this is useful 
even when there is weak correlation in the noise samples, the Locally Optimum Bayes Detectors and 
Estimators denved from it are not strictly optimum nor is the associated performance. Moreover, 
although independent time samples are easy to obtain, independent spanal sampling is more difficult, 
because of the strongly correlated nature of most noise fields. 

Efforts to overcome this difficulty have been made by a number of authors recently. A more general 
analysis, based on "moving-average” noise models, following the results of the author, has been carried 
out by Maras, using fully probabilistic criteria of performance. However, his analytical results do not lead 
readily to numcncal results for performance and algorithms. The present paper overcomes these 
limitations, particularly for the common cases of weak correlation in the noise samples 0(^ 0.2-0.3), by 
exploiting the usual Toplitz nature of the noise covariances. Added new features are various explicit exact 
results in important special cases. Both broad- and narrow-band signals and noise arc treated. 
Calculations show typical improvement is 0(2 dB) with sample correlations 0(0.2), or 0(10) in error rates. 
(Work supported by ITS/NTIA of U.S. Dept, of Commerce, Boulder, CO, 80303.) 
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SESSION ThP2 


COMMUNICATION NETWORKS 


Code Combining with Convolutional Coding and Sequential Decoding for CDMA Slotted Networks 
T. Ketseoglou and A. Polydoros Department of Electrical Engineering, University of Southern 
California, Los Angeles, CA 90089-0272 

We apply the idea of code combining to CDMA slotted networks with convolutional coding and 
sequential decoding. Both hard and soft decision decoding are examined. In addition, we study two 
different decoding scenarios. Under the first, perfect channel state (side) information (number of 
simultaneous transmissions) is available at the receiver (the decoder operates under perfect channel 
match) while under the second scenario no such information is available (the decoder operates under 
channel mismatch). Our results indicate high improvement in the average utilization and delay, at the 
expense of modest increase in system complexity. 


Multi-Access Protocols for Metropolitan Area Networks 
Manocl A Rodngues AT&T Bell Laboratories, Holmdel.NJ 07733 

We propose two distributed multi-access protocols and show that they operate well even in the 
presence of propagation delays. We consider a slotted bi-direcuon Bus being shared by bursty users. Each 
user is assumed to know the direction in which to transmit a given message. A limited number of Hags 
can be send in the header of slots flowing in the direction opposite to transmission. In the fust protocol, a 
user sends a busy signal when us state changes to busy (i.e.. having something to transmit) and an idle 
signal when it becomes idle. In the second protocol, a user sends a busy signal when it changes to a busy 
state and an idle signal M slots later, where M denotes the size of the message to be transmitted Upon 
observing an idle signal, a user estimates the number of active downstream users. Slots are allocated to 
emulate a target service discipline (c g , round-robin or FIFO). We analyze our two protocols and the 
IEEE 802.6 MAN protocol, and show that the proposed protocols are more suited to MANS. 


Optimal Admission and Routing at a Simple Data Network Node 

Ioannis Lambadans Electrical Engineering Department, University of Maryland, College Park, MD 
20783 

A stream of messages reaches the buffer of a node in a communication network, according to a 
Poisson distribution with parameter A. The messages are then to be routed over two channels with 
different propagation times that are exponentially distnbuted with parameters p.\ and /ij where p\ is 
larger than /t 2 . Two controls are to be applied at the node. The first control concerns the admission of a 
message into the buffer while the second control is responsible for routing the buffered messages over the 
two channels We determine the optimal policy which minimizes an expected cost involving the weighted 
sum of a penalty for not admitting a message into the buffer and a penalty for the delay experienced by 
the messages already stored in the buffer. (This research was supported by the Systems Research Center 
at the University of Maryland under NSF grant CDR-88-03012.) 


Performance Evaluation of Multi-Access Strategies for an Integrated Voice/Data Packet Radio 
Network 

Mohscn Soroushnejad and Evaggelos Geramotis Department of Electrical Engineering & Systems 
Research Center, University of Maryland, College, Park, MD 20742 

The problem of voice/data integrauon in a random-access radio network employing the ALOHA 
protocol in conjunction with recursive "transmission control is investigated. Code division multiplexing 
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is used as a suitable modulation in a radio environment to decrease the effect of multiple-access 
interference. Multi access control strategies are introduced which take advantage of multiple-access 
capability of the CDMA channel to accommodate several voice calls simultaneously, while the data users 
contend for die remaining (if any) multiple-access capability of the CDMA channel. The retransmission 
probabilities of the backlogged data users are updated based on estimates of data backlog and number of 
established voice calls which are obtained from the side information about the state of channel activities. 
A two-dimensional Markovian model is developed for the voice and data traffic, with the data backlog 
and number of established voice calls representing the state of the system. Based on this model, the 
voice-call blocking probability, the throughput of both traffic types, and the delay of the data packets are 
evaluated and the tradeoffs between the parameters of different traffic types are quantified. 


Stability Analysis of Asymmetric, Limited Service, Polling Systems 

Ramesh Rao and Amir Behroozi-Toosi University of California, San Diego, La Jolla, CA 92093 

Three cyclic server strategies have been studied in the literature. They are the exhaustive service 
strategy the gated service strategy and the limited service strategy In the performance analysis of such 
systems much of the attention has been given to delay analysis of the system and the system is usually 
assumed to be stable or in a stationary state. With gated or exhaustive service strategies, if one of the 
queues is unstable then all other queues become unstable too. This is not the case for the limited service 
strategy. 

In this woik we present a method for denying the stability conditions of limited service systems by 
combining a result from Lyones and a technique of constructing a sequence of auxiliary systems which 
dominate the actual system in a well defined sense. The stability region of the auxiliary system is derived 
without resorting to stationarity assumptions. Thus we are able to derive inner bounds to the stability 
region of the actual system. 


Avoiding Third Order Intermodulation Interference in Mobil Radio Systems 

Torleiv Klovc Department of Informatics, University of Bergen, Thorrwhlengsl. 55, N-5008 Bergen, 

Norway 

We consider mobile radio system for a collection of areas, and without third order intcrmodulation 
interference within each area. The problem has been considered by a number of authors, and it requires 
the following construction: 

An (/,/>-set of Disjoint Distinct Difference sets (DDD) is a set 

A=(A,.A 2 . A,) 

where 


A, = (a l; | 1 £ y £./) for 1 £ i £ / 

are disjoint sets of positive integers such that, for each i, all the differences a, t - a,/,, with j'* j are 

distinct 

Let 

A = A(A)-max (o v | 1 £i £/, 1 £; £/). 

For the application we want an (/./j-DDD with A as small as possible Let 
//(/,/) = min (A(A) | A is an(/,/)- DDD). 

Cicariy H ( l,J ) 2 U. We will present constructions of DDD which shows that H (1,J) = // for / > 4 J. The 
construction is based on a gcncralizauon of sonar sequences 
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Delay and Throughput in a Frequency-Hop Communication Network 

Sang Wu Kim Korea Institute of Technology, 400 Kusung-dong, Yusung-gu, Taejon 305-701 , Korea 

The packet delay measured in average number of retransmissions before the successful transmission 
of a packet in a slotted frequency-hop packet radio network is considered. Reed-Solomon coding is 
employed to correct the errors or the erasures occurring due to the multiple-access interference. We 
obtain the packet delay in terms of code rate and channel traffic, and determine the constraint on them for 
the delay to be less than a specific value. We also derive an accurate approximation to the constraint. The 
throughput of the network along with the constraint on delay is investigated to see the tradeoffs in the 
parameters of the network. Both side information and no side information cases are considered. 

It is observed that the delay increases abruptly after some channel traffic, but is shown to be kept 
wrthin a bound by controlling the code rate. This indicates that we can make the network stable by 
controlling the code rate according to the channel traffic. The delay performance ts found to be 
tremendously improved by using srde information at the receiver, and this improvement becomes more 
signrficant as the channel traffic increases. The maximum channel throughput is found to be attained at 
very low values of delay (typically when delay SI), and sacrificing the delay does not necessarily 
increase the channel thiougput. 


A New Decoding Scheme for Convnlutionally Coded ARQ 

T. Hashimoto Dept, of Information Sciences, Tokyo Denki Umv , Hatoyama, Saitama 350-03, Japan 

The Viterbi algorithm finds out the maxtmum likelihood path by comparing, at each step and for 
each state, paths that merge on the given state (or node) in the trellis diagram. The algorithm tends to 
make an error when the likelihood of the correct path is as small as incorrect ones due to channel noise. 

Known ARQ schemes based on the Viterbi algorithm employ devices which request the 
retransmission until one path, which is supposed to be the correct one, shows a sufficiently greater 
likelihood than the remainders Since those ARQ schemes can not detect errors occurred in the past, the 
error probability depends on how incorrect decisions are prevented at each step. Thus, efforts to attain 
higher reliability inevitably increase the probability of retransmission. 

In this paper, we propose a new ARQ scheme based on the generalized Viterbi algorithm (GVA) 
and show that the same reliability is attainable with the same complexity, but with less retransmission 
probability Actually, we can make the error probability arbitrarily small by increasing the length of the 
tail appended to each frame without substantially increasing the decoder complexity 


A Sub-Optimal Distributed Self-Organizing Mobile Radio Network Algorithm 
Yong Li and Bing-Zhcng Xu South China University of Technology, Guangzhou 510641, P R. China 
In this paper we have proposed and analyzed a sub-opumal distributed algorithm for self- 
organization networks, which can organize a set of scattered, mobile, radio-equipped nodes into a 
connected network. The algorithm has the ability to reorganize the nodes into a new network under very 
unfavourable circumstances (jamming, wartime conditions). This feature can improve the survivability of 
the network. It uses network topology information obtained from the partial connectivity matrix that can 
yield to fewer control nodes and which can reduce the overhead traffic and improve the responsiveness of 
the network to changing traffic conditions. Using Slottcd-ALOHA as channel access scheme, we analyzed 
and compared the original and new proposed algorithm, and found the throughput of the sub-optimal 
self-organization network to be better. The computer simulation of the algorithm has also been given. 
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Analysis of a Modified Hebbian Rule 

Ph. Pirct Philips Research Laboratory. Av. Van Becelaere 2. Box 8. B-1170 Brussels. Belgium 

Let X be an mxn matrix over U = {+ 1.-1} and associate to X its Hebb matrix H =X T X. Let U Q 
be the set { + 1,0,-1). It is well known that H can be used to define a mapping (/"-» Uq :yw sgn (yH), 
where sgn denotes the signature of its argument. In this paper one analyzes the properties of a modified 
mapping (/* -* Uq : y t-> sgn (yC ). where C is itself the signature sgn (//) of H. Assuming that (m -1) is 
close to (n-iyj/log(/t-l) on determines first for which values of fi the mapping yn>sgn (yG) 
stabilizes the rows of X. In particular, one shows that 

for any fi £0. the (r.j) entry of sgn (XG) is most often equal to the (r.s) entry of X. 
for/J<l/jr. thcr* row of sgn (XG) is roost often equal to the r* row of X, 
for /J < 1 l2x. the matrix sgn (XG) is most often equal to X. 

Then one obtains upper bounds on the (mean) residual error probability when the mapping 
y sgn (yG) is used to map y on the row of X that is closest to y. Although there does not exist any r > 0 
such that for r. -* «* one guarantees that aU errors in [rnj positions of a row of X are correctable in one 
step, one can prove that most of them are correctable by the mapping y sgn (yG) when r is smaller 
than a positive quantity, which is a function of fi. 


An Iterative Learning Algorithm That Updates Only When It Learns 

S. C. Huang and Y. F. Huang Department of Electrical and Computer Engineering. University of Notre 
Dame. Notre Dame. IN 46556 

The primary objective of this paper is to investigate a learning algorithm, recently proposed by 
Huang and Huang, for perceptron-like networks, the main feature of this algorithm so-called SUBPA is 
that it implements a selective learning strategy, thus updates only when it can learn from the input data. 

Results of convergence analysis for single layer, as well as multi-layer, networks will be presented. 
It is shown that, under very general conditions, the algorithm converges after a finite number of updates. 

An associative memory machine (AMM) is devised here using perception networks. This machine 
is novel as it is constructed with perception networks, as opposed to Hopficld networks. In addition, it can 
be used to demonstrate the capability of the aforementioned learning algorithm. The notion of global 
realization is introduced here, h is shown that SUBPA is particularly suitable for applications to global 
realization. The AMM is also used to demonst ra te the class of problems for which the back propagation 
algorithm can be suitable. Furthermore, applications to data compression for image processing purposes 
using pcrccptron-like networks are also demonstrated. 


Image Prediction for Error Concealment Using Neural Networks 

Nobukazu Doi. Toshiaki Takahashi. and Hideki Imai Centred Research Laboratory. Hitachi. Ltd.. Token 
Works, Hitachi. Ltd., and Faculty of Engineering. Yokohama National University. Japan 

An image prediction method is proposed for error concealment using a neural network. A (48.9.1) 
three-layered feed-forward network is chosen. The input layer is connected to pixels around an erroneous 
pixel to be concealed. The output layer has one unit and puts out a signal that is predicted to be capable of 
concealing the erroneous pixel. 

Using several images as training input and output data, the back-propagation learning procedure 
adjusts the network’s link weights to achieve ideal predictions. Image simulation confirms a neural 
network is better than the conventional linear prediction method, especially for high-frequency images. 
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However, there is a possibility of errors occurring in pixels connected to the input layer’s units. To cope 
with such cases, a hybrid neural network is proposed that uses signals obtained by the linear prediction 
method instead of erroneous pixel signals. It's confirmed that the hybrid method performs better than a 
linear filter only. 


Neural Network Applications for Jamming State Information Generator 

Lawrence T. Schaefer and Hyuck M. Kwon Lockheed Missiles and Space Center Company, Sunnyvale, 
CA 94088 and Houston, TX 77058, and Electrical Engineering and Computer Science Department, The 
University qfWiscortsin-Milwaukee, Wl 53211. 

A previous jamming state information (JSI) scheme, which was proposed for a coded frequency- 
hopped MFSK (FH/MFSK) system under partial-band noise jamming plus additive white Gaussian noise, 
utilized the maximum a postenori (MAP) rule based on the total energy received in the Af-tone signaling 
bands. It was assumed that the knowledge of partial-band noise jamming fraction was available to that JSI 
generator. Because this scheme reduces the Af-dimensioni information contained within an M- 
dimensional signal vector into one dimension, i.e., the total energy, the generated JSI may not be the best 
In addition, this scheme is hard to implement because the MAP rule needs the M-l order modified 
Bessel function In this paper we present a neural network approach to the JSI generation which is not 
only implementable but also uses the innate characteristic of the Af-dimensional vectored data to create 
the JSI. The efficiency of the new JSI generator with kr wn partial-band noise jamming traction will be 
compared with the MAP generator. Afterwards, the neural network scheme will be generalized to increase 
its robustness by allowing for an unknown partial-band noise jamming fraction. We find that the neural 
network JSI generator with or even without knowledge of jamming fraction can improve significantly the 
performance of a coded FH/MFSK communication system over the MAP JSI generator for high code rate. 
For example, the neural network JSI generator without knowledge of jamming fraction is 1.43 dB better 
than the MAP JSI generator in the bit energy-to-jamming noise power spectral density ratio required to 
achieve the cutoff rate of 0 7, for FH/binary FSK with the symbol cnergy-to-thermal noise ratio 15 dB. 


An Optimal Neural Net Model for Image Coding in The Position-Frequency Space in the Presence 
of Noise 

A. M. Elramsisi and M. A. Zohdy School of Engineering and Computer Science, Oakland University, 
Rochester, Ml 48309-4401 

Image representation in the frequency-position space (joint domain) is investigated based on the 
optimal properties of the Gabor basis functions (GBF’s) A major intent of this article is to embed the 
joint domain optimal features of GBF’s in the structure of neural netwoiks for image representations in 
the presence of noise The voltage controlled oscillator (VCO) neuron model is first introduced. Its joint 
domain neuron characteristic function (NCF) is analyzed through the mvesugation of the Kolmogorov-. 
Smoluchowski steady state differential equation (K-S DE) of the model for an optical input signal. At 
higher values of signal-to-noisc ratio (SNR), the NCF funcuon of the VCO neuron model resembles the 
GBF, and reflects their joint domain optimal properties. The range of resolution and consequently the 
trade-off between the spatial domain and frequency domain resolutions is now dictated by SNR values. 
The proposed neuron model is then encompassed in a three-layered neural network for image 
representation in the joint domain. The scheme is later expanded to account for nonuniform sampling as a 
characteristic feature of human vision. The performance of the proposed scheme is evaluated in terms of 
the fidelity in reconstructing the images after extracting the coefficients of expansion. Simulation results, 
modification to improve convergence of the network, and possible extensions for this work are also 
provided. 
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On the Stochastic Approximation Based Learning Algorithm for Neural Networks 
Valadimir Nedeljkovic and Milan M'losavljevic Institute of Applied Mathematics and Electronics, 
Belgrade, Yugoslavia, and School of Electrical Engineering, University of Belgrade, Belgrade, 
Yugoslavia 

In this paper we consider the RHW (rumelhart, Hinton, Williams) neural networks in the statistical 
pattern recognition tasks. We propose the stochastic back-propagation algorithm (SBP) as a modification 
of the well known error back-propagation algorithm (BP. The central part of our work is the proof of the 
theorem which gives sufficient conditions for convergence of the SBP algorithm, in mean square and with 
probability 1, to a local minimum of the criterion function. The final part of this wotk presents 
experimental results of comparison of SBP and BP using test examples suggested by Kohonen, Bama, 
and Chrisley. Comparative analysis of our experimental results with that reported by Kohonen, Bama, 
and Chrisley confirm superionty of SBP algorithm both in convergence rate as well as in accuracy of 
recognition. 


On Segmentation and Recognition of Connected Digits Based on Neural Network Model 
Jhing-Fa Wang, Chung-Hsien Wu, Ruey-Chinq Shyu, and Jau-Yien Lee Department of Electrical 
Engineering, National Cheng Kung University, Tainan, Taiwan, R.O C. 

In this paper, an automatic segmentation and recognition system based on a neural network model is 
proposed and back-propagation learning algonthm is employed to establish these networks. 

The main new idea for segmentation is to classify the energy, spectrum, and pitch-penod transitions 
that occur at the boundaries between syllables. These feature transitions are used as the input patterns of 
the neural network. The segmented syllables are then used as the basic units for recognition by feeding 
them into the well-trained network. 

Ten digits (0-9) and syllables spoken in Mandarin are used in the speaker-independent phase for 
segmentation experiments, and only ten digits are used in speaker-dependent phase for recognition 
expenments. With an average speaking rate at 150 per minute, the segmentation accuracy measured in 
coincidence rate of 95.7% and recognition rate of 97 2% can be achieved. 


Back-Propagation Neural Network Model Based On-Line Chinese Character Recognition System 
1-Chang Jou and Ching-Feng Hsu Telecommunication Labs , Ministry of Communications, P 0 Box 71, 
Chung-Li, Taiwan, R O.C., and Institute of Information & Electronics Engineering, National Central 
University, Chung-D, Taiwan, R.O C. 

A ncural-nctwork-bascd on-line Chinese character recognition (OLCCR) system is presented in this 
paper. The OLCCR system is able to recognize a Chinese character immediately after it is wntten on a 
tablet. The recognition process is composed of two phases' (I) the feature matching phase, in which a 
feature vector based on the stroke information is extracted, and (2) the character matching phase, m which 
the feature vector is matched with feature vectors of standard Chinese characters stored in the reference 
pattern database. In this paper, back-propagation neural-nets models, instead of conventional 
classification algorithms, are used to solve the pattern matching problems in CLCCR. Expenmentally, the 
time to recognize a handwmten character is 0.15 second on the average, and the recognition rate from 
73 3% to 94.4%, depending on the user's writing style. 
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A Zero Knowledge Proof Protocol for Communications Authentication 
Jonathan D. Low Verifone, Inc., 100 Kahelu Avenue, Mililani, HI 96789 

This paper describes a procedure for verifying the identity of a remote system. The confidence level 
of the verification approaches 100% rapidly with time or space. The algorithm uses only counting and 
parity testing and therefore executes effectively. The intelligence bits transmitted between the two parties 
have an equal probability of appearing as a 1 or 0 to the enemy. Hence, the transmitted bit sequences 
actually carry zero information. 


Communication Complexity of Secure Distributed Computation in the Presence of Noise 

Eytan Modiano and Anthony Ephremides Department of Electrical Engineering, The University of 

Maryland, College Park, MD 20742 

We consider a simple model of distributed computation that requires information exchange over a 
noisy channel. We utilize a communication protocol that requires alternate bit exchanges between two 
processors and are interested in determining the communication complexity of this exchange. First, we 
consider the case of a single public channel and compute the number of bits that need to be exchanged 
between the processors to permit 5-accuracy in their goal. For this computation we consider an error- 
detection-and-rctransmission mechanism of error control as well as an error-corrcction-and- 
retransmission mixture that are consistent with the logical protocol that governs this exchange. Second, 
we consider the case of the availability of an additional secret channel and arc interested in determining 
the minimum, number of bits that need to be exchanged over a secret channel in order to maintain 6 - 
uncertainty about the computation for an eavesdropper on the public channel. We consider various sub¬ 
cases under this case and obtain an upper bound on the number of secret bits when no error-control 
scheme is used. 


Modified Graham-Shamir Knapsack Public-Key Cryptosystem 

Chi-Sung Laih, Lein Ham, Jau-Yien Lee, and Yan-Kuin Su Department of Electrical Engineering, 
National Cheng Kung University, Tainan, Taiwan, ROC, and Computer Science and 
Telecommunications Program, University of Missouri-Kansas City, Kansas City, MO 64110 

Since Shamir first proposed the polynomial time attack to the basic Mc-klc-.lcllman knapsack 
cryptosystem, lately almost all knapsack cryptosystems based on the modular multiplications can be 
attacked. This is due to the fact that those systems must satisfy the constraint M > £" a„ where a, 's arc 
the deciphenng sequences and M is the modulus. In this paper, we propose a modified version of 
Graham-Shamir knapsack public-key cryptosystem By appropriately choosing the parameters, one can 
control the density of the public knapsack, which is the ratio between the number of elements and the 
maximum value in bits in the knapsack. It is easy to design a knapsack whose density is high enough to 
foil "low-density" attacks against our system. Moreover, the constraint of the modular multiplications 
does not require in the modified version We call this structure as the "partial structure" Therefore, we 
believe that no attacks can break this system in a reasonable amount of time at the moment. 
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On the Secure Communications of Group Oriented Societies 

Tzonelih Hwang Institute of Information Engineering, National Cheng Kung University, Tainan, 
Taiwan, R.O.C. 

A practical protocol based on the RSA scheme is developed for large group-oriented networks. The 
information sender does not have to specify who will receive the message because the members are 
anonymous and the organization policy of the receiving group is unknown. However, he is able to 
indicate the nature and destination of the message, e.g., how important the message is and in which order 
or to which department it will be sent. The receiving group can create its own policy on who can read the 
message according to the indication of the sender and the group's organization. 


Algebraic-Code Cryptosystems for Information Privacy, Reliability and Authenticity 

Tzonelih Hwang and T R.N. Rao Institute of Information Engineering, National Cheng Kung University, 

Tainan, Taiwan, R O.C, and University of Southwestern Louisiana, Lafayettt, LA 70504 

When confidential data are transmitted over communicational channels or stored in storage systems, 
one of the most important issues of concern is the assurance of their privacy, reliability and authenticity 
(integrity). Algebraic codes have long been proved to be very powerful in correcting noise errors. In this 
paper, algebraic-code encoding is combined with block chaining encryptions into one to obtain data 
secrecy, data reliaLJity and data authenticity simultaneously. These schemes can be implemented very 
efficiently, thus are very practical to be used in computer systems. 


Unbiased Block Substitution 

Lothrop Mittcnlhal Teledyne Electronics, 649 Lawrence Drive, Newbury Park, CA 91320 

A block substitution is a one-to-one mapping of the n-bit binary numbers onto themselves. Such 
substitution or transformation can be represented by a permutation. It is shown that certain permutations 
of the n-bit binary numbers define a block substitution by modulo 2 addition of one permuted set of 
numbers to another. Such permutations arc termed replicative A subset of these have an additional 
feature which is that the equations which they define have an additive relationship when viewed as 
vectors. Such permutations arc termed additive. An unbiased block substitution is defined It is shown 
that substitutions defined by an additive permutation arc unbiased and that any unbiased block 
substitution can be represented by a replicative permutation. Additive permutations are shown to form 
groups which retain the same properties. The conditions for existence of these additive permutai.ons are 
established, some properties of the groups determined, and the number of such groups enumerated and 
compared with all possible permutations of the n-bil numbers. 


Cryptographic Interleaving 

Sami Haran Universite'de Toulon el du Var, Faculte'des Sciences et Techniques, Avenue de TUhiversite', 
83130 La Garde, France 


The Relationship Between MDS Codes and Threshold Schemes 

Yang Yi Xian Dept, of Information Engineering, PO Box 145, Beijing University of Posts and 
Telecommunications, Beijing, 100088, PR of China 

The deep relationship between MDS codes and threshold schemes in cryptography is initially 
discovered in this paper We have found that by using the generator matnees or the panty-chcck matnccs 
of MDS codes, we can implement a new class of threshold systems in cryptography. 
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A New Measure for Stream Cipher Systems to Defend Against Correlation Attack and Linear 
Approximation Attack 

Zhang Muxiang Department of Applied Mathematics, Xi'dian University, Xi'an, People's Republic of 
China 

This paper shows that, in analysis and design of stream' cipher systems of several linear feedback 
shift registers with a nonlinear binary function, there exists not only a trade-off between the nonlinear 
order and the Correlation-Immune order of the nonlinear binary function, but also a trade-off between the 
Correlation-Immune order and success-rate of linear approximation to the nonlinear binary function. In 
order to avoid these trade-offs, a new measure for stream cipher systems to defend against Siegenthaler’s 
correlation attack is presented in this paper. 
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Non-Equiprobable Signaling on the Gaussian Channel 

A. R. Calderbank and L. H. Ozarow Mathematical Sciences Research Center, AT&T Bell Laboratories, 
Murray Hill NJ 07974 (40 min.) 

Many signaling schemes for the Gaussian channel are based on finite-dimensional lattices The 
signal constellation consists of all lattice points within a region R, and the shape of this region 
determines the average signal power. In the limit as N -> the shape gain of the W-sphere over the N- 

cube approaches — = 1.53 db. We show that the full asymptotic shape gain can be realized in any fixed 
6 

dimension by non-equiprobabie signaling. We desenbe shaping schemes that achieve a significant 
fraction of the available asymptotic shaping gain. The peak to average power ratio of these schemes is 
superior to that of cquiprobable signaling schemes based on Voronoi regions of multidimensional lattices 
the new shaping schemes admit a simple staged demodulation procedure 


The Merit Factor of Binary Sequences Related to Difference Sets 

Jim M. Jensen, H. Elbrdnd Jensen, and T H. HSholdt Mathematical Institute - 303, The Technical 
University of Denmark, DK-2800 Lyngby, Denmark 

The merit factor of long binary sequences constructed from cyclic difference sets are investigated. It 
is shown that only sequences arising from the Hadamard difference sets, that is 
(v.ifc, ) = (r5-1,2r-1, r-1), can produce long binary sequences with a nonzero ment factor. Following 
Baumert, Hadamard difference sets can be classified according to the parameter v. 1) v = 2" -1: Singer- 
and GMW-diffcrcncc sets. 2) v a prune: The quadratic residues modulus v. The Hall-sets if v = 4x 2 + 27. 
3) v =p Ip + 2): The twm-pnmc difference sets. The sequences arising from the Singer difference sets are 
the maximal-length linear shift register sequences (or m - sequences) The sequences constructed from the 
quadrahe residues are the Legendre sequences. It has been shown that the maximal merit factor of m - 
sequences, Legendre sequences and twin-prime sequences arc 3, 6 and 6 rcspecuvely. 


Lower Bounds to Moments of List Size 

Erdal Ankan Department of Electrical Engineering, Bilkent University, P 0 Box 8, Maltepe, Ankara, 
06572, Turkey 

The list-size random variable L for a block code is defined as the number of incorrect messages that 
appear to a maximum-likelihood decoder to be at least as likely as the true message. Lower bounds to the 
moments of L are of interest in a number of applicauons, particularly in lower-bounding the moments of 
computation in sequential decoding. We prove the following bounds for the list-size L belonging to a 
code with M codewords and blocklcngth N 

E (L‘) 2 K‘P mm {M,N,K) 

E{.L‘)in'(MIK)'P m „(K,N,n) 

where t £ 0 is an arbitrary real number, n a 1 is an arbitrary integer, K is an integer between 1 and M, and 
P mm (M,N,K) denotes the minimum, over all codes with parameter (M,N), of the probability of hst-of-K 
decoding error. We then prove by applying sphere-packing lower bounds to the first inequality above that, 
for any t i 0, 

£(L 5 ) > cxp[tN (R-R,)-o(N)] 
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where o (N) is a quantity that goes to zero as N goes to infinity, and 


ft, 


= (l/f)max-ln 
Q 


£ 


EGO')/’O'I ') 1 ' 0 " 1 
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where the maximum is over all probability distributions on the channel input alphabet, and the sums are 
over the channel input and output alphabets, respectively. For sequential decoding, this implies that the 
r’th moment of computation is unbounded at rates above ft,, for all r > 0, and settles a long-standing open 
problem. 


Nonuniform Sampling Theorems 

Michael David Rawn Dept of Mathematical Sciences, Manchester College, North Manchester, Indiana 
46962 

This paper takes as point of departure the work of Paley and Wiener, and later Levinson, on 
nonharmonic Fourier series, to denve nonuniform sampling methods. One theorem yields an extension of 
the work of Linden and Abramson on sampling using derivative samples. Some basic results of spline 
theory and Riesz bases in Hilbert Space are used in conjunction with Levinson’s work on gap and density 
theorems to obtain this nonunifoim sampling method. In another direcuon, we present a theorem on 
nonuniform sampling using entire interpolating functions that are Lebesgue square intcgrable. This 
sampling theorem demonstrates - among other things - the stability of the Bcssel-typc sampling 
expansions first discussed by Kramer, and by Campbell. General results on closure and completeness of 
sequences of Bessel functions obtained by Boas and Pollard, identities involving the beta function, and a 
transformation relating Fourier and Hankel transforms to one another are used in denying this 
nonuniform sampling method. 


On the Converse Theorem in Statistical Hypothesis Testing 

Kcnji Nakagawa and Fumio Kanaya NTT Systems Laboratories, 812C,, 1-2356, Take, Yokosuka-Shi, 
Kanagawa-Ken, 238-03 Japan 

The converse theorem of simple statistical hypothesis testing is investigated according to the 
Neyman-Pearson theorem and the differential geometry of the space of probability distributions on finite 
atoms. We obtained the result that the condition r > D {p \ ||po) can be divided into two cases, and a lucid 
explanation is given for Han and Kobayashi’s linear funcuon /,() We also mention general upper bounds 
on the first-kind error probability. These results can be extended to Markov sources. 

Let p, be the +1 geodesic connecting p 0 , p t and let p„ = lim p,. We call D(p x \\p 0 ) the upper 

bound of the exponent of the first-kind error probability. We obtain the theorem, if r > D(p„\\p 0 ), the 
power exponent is r + D(pdAp i)-D(pJ\p 0 ), which equals },(■) 


Optimization of Signal Sets for Partial-Response Channels 

Michael L. Honig, Kenneth Stciglitz, and Stephen Norman Bellcore, 445 South St, Morristown, NJ 
07960; Dept, of Computer Science, Princeton, University, Princeton, NJ 08854, and Information Systems 
Laboratory, Stanford University, Stanford, CA 94305 

Given a linear, time-invariant, discrete-time channel with impulse response A [it], we consider the 
problem of constructing N input signals of finite length T that maximize minimum / 2 distance between 
pairs of outputs. Two constraints on the input signals are considered: a power constraint on each of the N 
inputs (hard constraint), and an average power constraint over the entire set of inputs (soft constraint) 
The hard constraint problem is equivalent to packing A points in an ellipsoid innnn(f,JV-l) dimensions 
to maximize die minimum Euclidean distance between pairs of points 
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We describe gradient-based numerical techniques that are used to find locally optimal solutions to 
the preceding signal design problems with both hard and soft constraints. In the case of hard constraints, 
feasible descent directions are found by solving linear programs. For the soft constraint problem, we 
maximize a penalty function that approximates the minimum-distance cost function. The penalty function 
is similar in form to the error criterion used by Foschini, Gitlin, and Weinstein, where two-dimensional 
signal sets are optimized for a nondispersive channel with additive white Gaussian noise. 

Numerical results, consisting of minimum distance vs. input length for different information rates, 
are given for the soft constraint problem. The channels considered are the identity channel, the 1 -D 
channel, and the 1 -D 2 channel. The computer-generated signal constellations are superior to multi¬ 
dimensional constructions based on dense lattices when the number of points per dimension is small. 

Our numerical results are compared to upper and lower bounds on e-capacity, which for a given 
linear time-invariant channel and information rate gives the maximum achievable minimum distance, or 
coding gain, as the input length tenjs to infinity. Results indicate that asymptotically, the maximum 
coding gains for both the (l-D)Z'u and (1 -0 2 )f5 channels are approximately 1 dB less than the 
maximum coding gain achievable fbi the identity channel at a rate of 1 bit/sample, and 2 dB less at a rate 
of 2 bits/sample. 


On Randomization in Communication Complexity 

King Fai Pang Systems Research Laboratory, LSI Logic Corporation, 4400 Bohannon Drive, Suite 230, 
Menlo Park, CA 94025 

Processors A and B have n-bit binary sequences x and y respectively. They communicate with a 
protocol, over a noiseless channel, so that at least one of them computes the value of a function f(x,y) for 
all xe X and ye V. We show that when rnors are allowed for a small portion of the inputs, 
randomization only reduces the number of bits required substantially in the worst case. Specifically, we 
prove an exponential gap between worst case randomized and deterministic complexities The average 
case complexities, on the other hand, only differ by at most a constant factor. 
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Error Evaluation for Nonbinary BCH Codes by Lagrange Interpolation 
C.P.M.J. Baggen Philips Research Laboratories, P 0. Sox 80 000,5600JA Eindhoven, The Netherlands 
In this paper an alternative is given for Forney’s method of error evaluation that is encountered in 
time-domain decoding of nonbinary BCH codes. This new method avoids the division needed at that 
stage of decoding. It turns out that a Lagrange polynomial L(z) can be found such that for all 
crroriocations X„ the error value Y, satisfies : L(2f7*) = K l . Furthermore, L(z) can be determined using 
Euclid’s algorithm before the roots of the error locator are known. Therefore all divisions can be 
concentrated in a processor-like structure that also solves the Key-Equation, leading to reduced hardware 
implementations. 


Af-Adic Residue Codes 

Vanessa Job Department of Mathematics, Marymount University, 2807 Glebe Road, Arlington, VA 
22207-4299 

The m-adic residue codes, defined by Bmaldi and Pless, are a generalization of the quadratic residue 
codes. They exist at prime lengths p over fields GF(q) when m | (p -1) and (q,p) = 1. The m-adic residue 
codes are investigated. Included are the proof of the existence ot an infinite class of m-adic residue codes 
for each m, a construction of a subgroup of the automorphism group for each m-adic residue code, a 
theorem which gives rcstricuons on the form of the idempotents of the m-adic residue codes, and lower 
bounds on the minimum odd weights of the odd-like m-adic residue codes. In addinon, containment 
rclauons between the m-adic residue codes and the quadratic residue codes are demonstrated. 

Tables arc included which contain idempotents and minimum weights of the binary m-adic residue 
codes of lengths less than or equal to 127 Many of these codes have the highest possible minimum 
weights known for codes of their lengths and dimensions 


Legendre Sums and Weights of QR Codes 

Tor Hellcselh Department of Informatics, Thorrmhlensgt 55, N-5006 Bergen, Norway 

We will present a connection between Legendre sums and the weights of the codewords in a 
circulant code related to a quadrauc residue (QR) code. Let F = GFlp), p a pnmc, denote the finite field 

with p elements. Let (—) denote the Legendre symbol defined by 
P 


(^> = 
P 


1 

-1 

0 


■f xeQ 
ifxe N, 
if jc = 0, 


where Q is the set of squares and N the set of nonsquares in F Let E{x) = V „ x q , and let C be die 

linear circulant code of length p generated by E (x) and it cyclic shifts. An arbitrary codeword in this code 
can be wntten as 


c(x) = x h £(*)+ ■ + x 1 ' E(x) (mod^-l) 

where 0S;i< • < y, < p 
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Main theorem. Let w denote the weight of the codeword c(jt), then 


w = 'A 


p+(-ir‘ [k—j-zc—) 

[teF P i*l P 


where/(r) = ('-A) and g,(t) = 


m 

t-i, 


The results are proved using methods for solving a system of equations in finite fields and by using 
Gaussian sums. 


Bandwidth Efficient Concatenated Scher’- for Fading Channels 

Branka Vucetic Electrical Engineering Department, The University of Sydney, Sydney, Australia 

Error control coding can greatly improve the performance and extend the range of fading channels. 
An important feature of concatenated coding schemes is that very low error rates can be achieved with 
reasonable complexity of decoding. Concatenated codes consisting of short constraint length trellis inner 
codes with different number of states and Reed-Solomon outer codes are considered to achieve large 
coding gains with small bandwidth expansion in the presence of frequency-nonselective slow Rayleigh 
and Rician fading. Both errors-only and errors-erasures decoding algorithms for outer codes are applied. 
Upper bounds on bit error probability performance in the presence of fading are obtained and compared 
with simulation results for zero channel memory. The effect of interleaving in eliminating channel 
memory is investigated. The performance gains that are achieved by the coding scheme relative to the 
reference uncoded systems are illustrated via some examples. Results are obtained both by analytical 
methods and computer simulation 


Balanced Binary Pseudorandom Sequences with Low Periodic Correlation 
Shinya Matsufuji, Kyoki Imamura, and Sucyoshi Soejima Department of Electrical Engineering, Saga 
University, Saga, 840 Japan, and Department of Computer Science and Electronics, Kyushu Institute of 
Technology, hzuka, Fukuoka, 820 Japan 

The families of binary (0,1) pseudorandom (PR) sequences with low periodic correlation 
properties necessary for a spread-spectrum multi-access communication system have been known such as 
Gold, Kasami, bent-function and No-Kumar sequences. These sequences, except the bent-function 
sequences, are not balanced, i e, the maximum value of the sequence imbalance (the difference between 
the number of l’s and the number of 0’s in one period) is very large. 

In this paper a new family of 2” n binary PR sequences of period 2" — 1, where n is even, is 
introduced and shown that every sequence within the family is balanced, i e, the sequence imbalance is at 
most 1 


Periodic Correlation Function of the Bent-Function Sequences 

Noriyuki Koga, Shinya Matsufuji, Kyoki Imamura, and Sueyoshi Soejima Dept, of Elec Eng . Saga 
University, Saga, 840 Japan, and Dept of Comp. Set and Elec, Kyushu Institute of Technology, hzuka, 
Fukuoka, 820 Japan 

Bent-function sequences of period 2" -1, where n = 0(mod 4), are balanced binary {0,1) sequences 
and have a large linear span. However, the explicit formula have been not known for their periodic auto 
and cross-correlation functions. Especially there have been no study on the number of phaseshifts at 
which the correlation funcuon takes the same value. This paper gives the explicit formula for the value 
distribution of periodic correlation function of the 2' n bent-function sequences (/ A (a‘) | 0Si£ 2"-2, 
Ae V, n \ defined by / A (r) = g A (u)+ tr*(ax) and g A (u) = u'[U 2 + A(u 2 ) + A'u, where a is a pnmilive 
element of GF( 2"), V k the set of all binary column vectors of dimension k, u' = [u'f u^J, with 
Ui, U 2 eV,/i, a column vector whose i-tfi element is defined as trJ(J} t x) using a basis [0i,Pi, .,P*n\ for 
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GF(2 nl2 ) over GF( 2), rrj() the trace from GF( 2") to GF(2), h() an arbitrary function mapping V„ /4 
into Vj, oe G/ r (2"K?/ r (2"' 2 ), and the superscript t denotes the transposition. 


Improved Balanced Encoding 

R. M. Capocelh, L. Gargano, G. Landi, and U. Vaccaro Dipartimento di Matemattca G. Castelnuovo, 
Universita'di Roma “La Sapienza", 00185 Roma, Italy, and Dipartimento di Informatica ed Applicazioni, 
Universita'di Salerno, 84081 Baromssi (SA), Italy 

A binary code of length n is called balanced if each codeword contains [n/2l (or|/t/2j) l’s. 

Balanced codes are useful for the detection of unidirectional errors, that is, errors that can transform 
0 into 1 but not 1 into 0 (or vice versa). Since each error changes the number of 1 ’s in the codeword, 
balanced codes are all-unidirectional error detecting Recent results show that many faults in VLSI 
circuits produce unidirectional errors. All-unidirectional error correcting codes are also useful for 
preventing subversions of the information recorded on ‘write-once’ memories, like digital optical disks. 

Knuth first afforded the problem of constructing balanced binary codes ha.ing efficiently 
computable encoding and decoding functions and proposed both sequential and parallel efficient 
encoding/decoding schemes. Subsequently Al-Bassan and Bose improved Knuth’s sequenual encoding 
scheme. They show that it is possible to encode all binary sequences of length k with a balanced code of 
length k+piSki 2 pH -p -2. 

In this paper we improve the construction proposed by Al-Bassan and Bose, that is, for a number p 
of parity bits and a number k of information bits, where 


k= 2 p+l -2x-4 ifp is even, and k = 2 p * l -2x-5 


we prove that, if x satisfies 


M+x 

» E 

i«U>/2J-x 


p 

\pn\-x-\ 

’ * 

p 

i 

a E * 

i-i 



ifp is odd, 


it is possible to encode all k-bits information sequences into a balanced code of length k+p. Relauon (1) 
is satisfied at least for x 2 . Evaluations show that there are values of x less than v p that satisfy relation 

(I) and suggest that a better bound might exist. 

The encoding and decoding procedures are based on the sequential complementation of bits and 
have time and space complexity 0 (k) 


Decoding of Algebraic Geometry Codes 

J. Justescn, K. J. Larsen, H. Elbrond Jensen, and T Haholdt Institute of Circuit Theory and 
Telecommunication and Mathematical Institute, the Technical University of Denmark, DK-2800 Lyngby, 
Denmark 

Based on our earlier results on the decoding of codes from algebraic geometry we present some 
improvements of the algorithm. For a code of length n and designed distance d, complexity is 0(n 2 ) and 
most error patterns of weight < d/2 are corrected. We present a class of error patterns which was not 
corrected by previous algorithms, and discuss decoding methods for these cases. 
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Investigation on a New Class of Bilateral-Checking Codes 

Jin Fan, Fan Pingzhi, and Chen Zhi Dept, cf Computer Science & Engineering, Southwestern Jiaotong 
University, Emei, Sichuan 614202, People’s Republic of China 

Based on the Block Design theorem, a new kind of Bilateral-Checking Code is suggested in this 
paper. Just as its "ante sounds, "Bilateral-Checking'' means tnat all die information bits are checked by 
the check bits, and conversely, all the check bits are checked by the information bits. 

We show that under some given restrictions, Bilateral-Checking Codes can easily be derived by 
means of a symmetric balanced incomplete block design. Since this new class of linear codes is 
majority-logic decodable, a rather high decoding speed may thus be expected. 
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SESSION ThP7 


ERROR-CORRECTING CODES H 


A Family of Efficient Burst-Correcting Array Codes 

Mario Blaum IBM Research Division, Almaden Research Center, San Jose, CA 95120 

We present a family of binary burst correcting array codes defined as follows: consider an n\ xn 2 
array with ni = 4u + v + 2 and n 2 = 6n + 2v + 5, u 5 1, v 20. v * 1, each row and column having even 
parity. The bits are read diagonally starting from the upper-left comer. The columns are viewed 
cyclically, i.e., the array is a cylinder. If one diagonal has been read out, proceed with the second diagonal 
preceding it. 

We prove that codes of this type can correct any burst of length up to »j. The burst-correcting 
efficiency of this family tends to 4/5 as As a comparison, the burst-correcting efficiency of other 
families of array codes tends to 2/3. the same is true for Fire codes. 

We also present a simple decoding algonthm for the codes defined above. 


A New Burst and Random Error Correcting Code: The Projection Code 

Gary R. Lomp and Donald L. Schilling SCS Telecom, Inc, Port Washington, NY 11050, and City 
College, Department of Electrical Engineering, City College of New York, New York, NY 10031 

A new coding technique, capable of detecting and correcting both random errors and burst errors is 
described. The technique is based on an extension of majority logic coding, and in fact provides an 
effective design methodology for constructing such codes. Decoding is accomplished by a simple 
iterative scheme, wherein the number of errors is reduced significantly with each iteration. Both the 
encoder and decoder are easily implemented as either a block code or convolutional code. Binary and 
nonbinary codes are available and erasure decoding is readily performed. Bit rates in excess of 100 Mbs 
are readily achieved using programmable gate arrays and 24COb/s encoding and decoding can be achieved 
using a microprocessor based codec. 

The decoding algorithm is capable of correcting many error patterns of weight in excess of the 
guaranteed minimum correction capability. It is shown how this feature may be effectively exploited in a 
hybnd code design that employs cyclic coding in a natural fashion within die present code. The resulting 
hybnd code has increased random error correction capability while retaining the burst error correction 
capability. 

This paper contains a discussion of the encoding and decoding processes and performance data as 
estimated analyucally and determined by simulation. Some details of the code structure and design of 
specific codes arc also given. 


Burst Asymmetric Error Correcting Codes 

Scungjm Park and Bella Bose Department of Computer Science, Oregon State University, Corvallis, OR 
97331-3902 

Codes capable of corrccung burst asymmetric errors are desenbed The proposed code needs 
approximately b + log 2 t + 1/2 logj log 2 * check bits to correct a burst of b asymmetric errors, where k is 
the number of information bus Thus when log 2 <fc + 1/2 log 2 logak < b, the proposed code is better than 
any burst symmcinc error correcting code. The optimality of the codes is also considered. 
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Burst-Error-Correcting and Detecting Codes 

Kumar N. Sivarajan, Robert J. McEliece, and Henk C. A. vanTilbotg California Institute of Technology, 
Mail Code 116-81, Pasadena, CA 91125 

In this paper, we ptesent several recent results in the area of simultaneous burst-correcting and 
detecting codes. We have obtained bounds similar to those of Reiger and Abramson for these codes, 
determined the true butst-conection and detection capabilities of Gilbert codes and developed efficient 
algorithms to determine the burst-correcting and detecting limits of cyclic codes. We also observe that 
many of the "good" burst-enor correcting codes have very little burst-detecting ability. On the other hand. 
Fire codes which have strong burst-conection and detection capabilities may prove to be optimal. 


An Application of Error-Correction Coding to Semiconductor Memories 
C.P.M.J. Baggen Philips Research Laboratories, P.0 Box80 000,56001A Eindhoven, The Netherlands 
In this paper, we introduce a particular concept of ECC that is tailored to semiconductor memories. 
The mam features of the considered class of ECC schemes ate the regular repetitive structures which 
facilitate design, and the short decoding delays. Furthermore, the choice of the decoder site allows the 
definition of long codewords, which are necessary for high-rate codes. 

The basic idea of the codes can be understood by considenng them as convolutional codes, where 
the time coordinate has been replaced by the space (column) coordinate. We discuss a particular class of 
codes which appear to be equivalent to codes proposed by Wyner and Ash. 

The considered class of codes appear to offer the required combination of low complexity, small 
decoding delay and high rate. 


On the Correcting Capabilities of Product Codes 

L.M.G.M. Tolhuizcn and C P.M.J. Baggen Philips Research Laboratories, P 0 Box 80 000, 5600JA 
Eindhoven, The Netherlands 

In this paper we show that a product code is much more powerful than is generally expected. 
Despite the poor minimum distance, product codes may still offer a good performance, even with 
relatively simple decoding algorithms because of the low number of low weight codewords. Firstly, we 
classify and count all codewords of weight less than d,d c + msx(d r d c ), where d r and d c refer to the 
distances of the row code and column code respectively. This leads to an upper bound on the number of 
low weight error patterns that a nearest neighbor decoder does not necessarily decode correctly. Secondly, 
we present a class of error patterns which have the all-zero word as closest codeword. This class suggests 
decoding possibilities beyond those already known for the simultaneous correcuon of burst errors and 
random errors. 


ECC for Multi-Valued Random Access Memories 

Rodney M. Goodman and Masahiro Sayano Department of Electrical Engineering, 116-81, California 
Institute of Technology, Pasadena, CA 91125 

A generalization of simple codes for decoding approximate errors, that is, those errors which result 
in values which are approximately that of the true values, will be introduced This situation arises when 
digital information is stored in analog levels and is corrupted only slightly in encoding or decoding, as in 
the case of multi-valued RAM. Such codes become important if data is to remain error-free as more bits 
are encoded in analog RAM cells and the size of the discrete steps which distinguish one digital data 
block from another decreases. In particular, errors of this type tend to occur in phased bursts due to timing 
or reference value errors, so phased burst errors must be dccodable, while to maintain high data density 
and access rates the codes must be high-rate and easily encoded and decoded 
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Upper and Lower Bounds on the Error Performance of Punctured Convolutional Codes 

Guy Begin and David Haccoun Department of Electrical Engineering, Ecole Polytechnique de Montre'al, 

P.O. Box 6079, station "A", Montre'al (Quebec) Canada H3C 3A7 

High-rate punctured convolutional codes are derived from low-rate convolutional codes by the 
periodic elimination of code symbols after encoding according to a perforation pattern. Powerful 
punctured codes may be selected vn the basis of bounds on the achievable error performance computed 
from the resulting characteristics (free distance, weight spectrum, distance profile, etc.) of the codes. 
Since the characteristics of a punctured code cannot be directly related to those of its original low-rate 
code, searching for powerful high-rate punctured convolutional codes involves a new evaluation of the 
characteristics for each combination of original code and perforation pattern. Likewise, ascertaining the 
error performance of a punctured code for a given coding rate R and memory M generally involves the 
full determination of its characterisucs. 

Fortunately, upper bounds relanng the original and punctured weights of remerging paths may be 
derived. Using these relations, an upper bound on the error perfotmance of a punctured code may be 
computed from the characteristics of the original code and the coding rate. On the other hand, using a 
completely different approach, a lower bound on the performances of punctured codes may also be 
denved. Based on the similanty that exists between perforation and erasures on an erasure channel, the 
performances of punctured codes with a random perforation pattern may be evaluated. Using classical 
arguments, these performances lower bound those of the best punctured codes. Comparisons between the 
lower and upper bounds and Viterbi's upper bound on error performance confirm the validity of the 
proposed approach for ascertaining the achievable error performance of punctured codes. 
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SESSION FA la 
MAGNETIC RECORDING 


Performance of Equalizers in Digital Magnetic Recording Channels 

John G. Proakis and Dennis Tyner Department of Electrical and Computer Engineering, Northeastern 
University, 360 Huntington Avenue, Boston, MA 02115 

The performance of linear, decision-feedback, and MLSE (maximum-likelihood sequence 
estimation) equalization techniques is evaluated for a digital magnetic recording channel. Both uncoded 
and run-length-limited (RLL) coded modulation are considered in the evaluation of the equalizer 
performance. Also evaluated is the performance of linear and decision-feedback equalizers which shape 
the desired signal pulse at the output of the equalizer into one of a family of partial response systems. 


The Capacity Region for Write Unidirectional Memory Codes over Arbitrary Alphabets 

W. M. C. J. van Overveld Dept of Electrical Engineering, Eindhoven University of Technology, P 0 

Box 513,5600 MB Eindhoven, The Netherlands 

Binary Wnte Unidirectional Memory (WUM) codes were designed for the rewritable optical disk. 
Writing is done in alternating 0- and 1-cydcs: in a c-cycle, only c’s can be written (ce (0,1)) Only the 
encoder knows the previous state of the disk. 

We generalize this to WUM codes over an arbitrary alphabet (0,1, ^-1}, where a c-cycle is 

followed by a (c + 1 mod <?)-cycle. With M c = the number of messages that can be written in a c-cycle and 
N.= the block length, we define R c .= log ( M c )/N and C,:= the capacity region, ie., the set of all 
achievable rate tuples (/? 0 . ^ t, Rq-t), We denve the expression for C,. 

For the symmetneal case, we find an elegant formula for the maximum C such that 
(C, C, ... C)E C,. we can wnte C in terms of the largest real zero of a polynomial We also prove that 
symmetneal rale R =(q -\)lq can be achieved with an easy code construction. 


On the Capacity of the Bit-Shift Magnetic Recording Channel 

S. Shamai (Shitz) and E. Zchavi Department of Electrical Engineering, Techmon - Israel Institute of 
Technology, Haifa 32000, Israel 

Tne bit shift (peak shift) effect is often encountered in magnetic recording and it rises mainly due to 
the effect of noise and read-out impairments and clock jittering We investigate the degrading effect of 
this phenomenon on the capacity of a binary (d,k) coded system, commonly used in magnetic recording 
The bit shift channel is best formulated in terms of phrase lengths. A phrase starts with none, one or more 
'O' and terminates with a single T, thus, any binary ('O', '1') sequence is uniquely decomposable into a 
concatenated sequence of phrases The shift effect, restricted here to no more than a single channel bit 
position, either shnnks the input phrase length or expands it by 1 We restrict our discussion to d > 2, thus 
additional phrases arc neither generated nor existing phrases phrases arc destroyed Keeping track of the 
inherent correlation of consccuuvc, bit shift affccred, phrase lengths we observe that: 

y. = x, + 






where x, stands for the /-th channel input phrase length and any y, is the corresponding channel output. 
The 1 i.d. ternary random vanable e, taking on (-1,0,+1) values designates whether a bit shift has occurred 
(e, =±l)ornot(£, =0). 

An increasingly tight and easily calculable sequence of upper and lower bounds on capacity 
(normalized per channel bit input) 

C= hm sup E(X N )' l l(Y N X N) 

~ — — 

is derived. Here X* 1, Y N stand respectively for input and output vectors of N consecutive phrase 
components, /(-, -) hniTH (•) stand respectively for the mutual information and entropy functionals, E is 
the statistical average operator and the supremum is taken over P (X N ) e P(d,k) - the probability measure 

of X N satisfying the (d,k) constraint, that is: x, e (d+\,d+2,..Jc+ 1 )i = 1,2. N. The capacity of a 

concatenated scheme in which the bit shift channel is followed by a binary symmetric channel is also 
addressed and certain upper and lower bounds are reported. 


Spectral Null Codes 

K. A Schouhamer Imminl: Philips Research Laboratories, 5600 JA Eindhoven, The Netherlands 

Often, the servo position information of magnetic tape or disk recorders is recorded as low- 
frequency components, usually called pilot tracking tones. To circumvent interaction dunng read-out 
between the pilot tones and the user information, user information is often encoded in such a way that the 
power spectral density function of the encoded stream vanishes at the pilot tone frequencies. Binary codes 
giving nse to a spectral null at an arbitrary frequency are used to provide space for the allocation of 
auxiliary pilot tones. 

This paper deals with encoding methods in which binary data are mapped into constrained binary 
sequences for shaping the spectrum. We compute the rate and power spectral density function of 
mcmorylcss codes that exhibit spectral nulls. The relationship between code redundancy and spectral 
notch width is quantified with a parameter called the sum variance. 



SESSION FAlb 


INFORMATION THEORY APPLICATIONS 
Necklace Properties of Shuffle-Exchange Graphs 

S. Song and E. Shwedyk Department of Electrical Engineering, University of Manitoba, Winnipeg, 
Manitoba, Canada, R3T 2N2 

Shuffle-exchange (SE) graphs, originally used as an interconnection methodology for parallel 
computations, have been proposed as a VLSI layout method for the implementation of the Viteroi 
algorithm and the Fast Fourier transform. Some asymptotically optimum VLSI-Iayouts are based on SE 
graphs. 

A key part of the layout technique is partitioning the graphs’s shuffle edges into necklaces. This 
paper presents results which show: 

a) under what conditions a particular shuffle-exchange network contains only full necklaces 
(excluding self-loops). 

b) when the graph has degenerate (nonfull) necklaces and an enumeration of their lengths 

c) a systematic method of listing all necklaces. 

It is shown that counting necklaces is independent of memory length and so enables the table 
construction which lists the number of binary necklaces for SE graphs of various memory length. 

Selection-Based Locally Connected VLSI Architectures for the (M,L) Algorithm 
E.M. Leiby III and Scshadri Mohan ECE Dept., Clarkson Uruvers’ty, Potsdam, NY 13676 

The ( M , L) algorithm has found applications in speech and image coding, decoding of 
convolutional codes, cpm codes, and trellis codes, and in the decoding of sequences received over 
intersymbol interference channels With its ever increasing appFcations, and to cope with the requirement 
of high data rates, there is an urgent need to design architectures for parallel processing suitable for VLSI 
implementation. Here we propose an SIMD mesh- and a tree-based architectures for the algorithm and 
show how the algorithm may be implemented on these. We compare these to two other architectures 
proposed in the literature, one due to Mohan and Sood and the other due to Simmons. 

Using Decision Trees for Noiseless Compression 

P A. Chou AT&T Bell Laboratories, Rm 2C-479, 660 Mountain Avenue, Murray Hill, NJ 07974 

We use a binary decision tree foi noiselessly compressing a Finite-alphabet stationary stochasuc 
process (X,). by testing, at each internal node, one of the k previous symbols X„_|„. ,X„_*, and by 
outputting, at each leaf, the distnbuuon of the current symbol X, Since the output distribuhon of X, 
depends on the previous k symbols, the tree models the conditional distribution of X„, and can be used to 
dnve a conditional entropy coder for noiseless compression. We build the decision tree by recursively 
applying, at each node, a recently developed algorithm for partitioning the alphabet of a discrete random 
variable X into two bins, such that the ~r> ditional entropy of a related random vanable T (given the bin) 
is minimized. Thus tnc tree is expires .resigned to minimize the conditional entropy of X„, given the 
leaves of the tree. The partitioning jrohm is equivalent to the X-means algorithm for pattern 
clustering, or to the generalized Llo>. Ugonthm for vector quantizer design, but uses Kullback's 
information divergence, or relauvc entropy, as the distortion measure Using this method, we reduce the 
bit rate of an image coder by 46 percent 


Lower Bounds on the Capacity of Asymptotically Good Spherical Codes in the Gaussian Channel 
Dejan E. Lazid Umversitat Karlsruhe, Institut fiir Algorithmen und Kognitive Systems, D-7500, 
Karlsruhe 1, Fed Rep of Germany 

The capacity of a sequence of a specific family of codes with the fixed bit code rate is defined as the 
least upper bound on permissible bit code rates, i.e. on bit code rates for which probability of error 
decreases to zero as the number of code dimensions grows beyond all bounds. For the family of 
asymptotically good spherical (i.e., equal energy) codes, a new lower bound on the code capacity is 
derived, and compared to those found in the literature. The new bound is greater than zero for signal-to- 
noise ratios below 0 dB, which was not a feature of the previous ones. 


SESSION FA2 


PATTERN RECOGNITION 


A Reformulation of the EM Algorithm for Hidden Markov Model Parameter Estimation 
M Ostendorf and J. R. Rohlicek ECS Department, Boston University, Boston, MA 02215, and BBN 
Systems and Technology Inc., Cambridge, MA 02138 

Hidden Markov models (HMM) are used in pattern recognition to represent time-varying signals, 
such as speech. A hidden Markov model consists of a discrete-time Maikov process representing an 
unobserved state sequence, with random observations which are conditionally independent given the state 
sequence. The focus of this paper is on the parameterization of the joint likelihood of the state and 
observauon sequences In particular, we present a parameterization which is not typically used and show 
that the maximum likelihood parameter estimate is related to a maximum mutual infoimation critenon 
between observations and states. This suggests a method for jointly estimating the parameters of a 
discrete hidden Markov model and an observation vector quantizer. A tree quantizer design algorithm 
based on maximizing the mutual information between the observations and the model states is presented. 
This quantizer has an additional advantage in that it is well-suited to the use of non-homogencous 
features. 


A Two-Dimensional Maximum Likelihood Approach for Image Edge Detection 
Jack Koplowitz and Xiaobing Lee Department of Electrical and Computer Engineering, Clarkson 
University, Potsdam, NY 13676, and SBC Technology kesources Inc, 550 Maryville Center Dr., St 
Louis, MO 63141 

Traditionally, the problem of edge detection in images has been to locate those pixels which contain 
an edge, i.e., a jump in the intensity level Recently a number of researchers have turned their attention to 
obtaining subpixcl accuracy in estimaung edge location. This work has generally focused on matching a 
parametnc surface to the region in the image where an edge occurs. Here a two-dimensional maximum 
likelihood approach is used to obtain the most likely position and onentation of the edge. Comparisons 
show that the maximum likelihood approach gives considerably less error in locating the posiuon of the 
edge over a wide range of noise condiuons, as well as blurring which may be unknown to the estimator. 


Approximate String Embedding in a Labeled Graph 

San Wei Sun and S. Y Kung Telecommunication Laboratories, Ministry of Communications, P 0 Box 
71, Chung-Li, Taiwan. ROC, and Department of Electrical Engineering, Princeton University, 
Princeton, NJ 08544 

This paper presents a dynamic programming to embed a given stnng within a directed, connected 
and labeled graph with flexible error correction metnc Common error correction operations, namely, 
omissions, insertions, substitutions, and the reversals, are tackled The proposed algorithms will find an 
optimal path in such a graph with a minimal cost of corrections. Two versions of dynamic programming 
algonthms are devised to find the optimal paths The embedding problems with omission, insertion, and 
substitution operations arc modeled by a trellis diagram, then a Vitcrbi algonthm is applied to find the 
best path in the trellis state transition diagram. For problems with omission, insertion, substitution and 
reversal operations, a linear embedding procedure is devised to rearrange a graph in a linear form, then a 
general dynamic programming algonthm is proposed to find the optimal path. The scheme is applied to 
an On-Line Chinese Character Recognition problem with a vanety of wnting senpts for a character, the 
cxpcnmcntal results have venfied the feasibility and cffccnvity of this approach 



Classification with a Reduced Complexity Nearest Neighbor Algorithm 

Tam is Linder and Gabor Lugosi Institute for Communication Electronics, Technical University of 
Budapest, Budapest Hungary 

In 198(5 Vidal presented an algorithm, which on the basis of some simple geometric considerations, 
finds the nearest neighbor in a metric space much faster than the exhaustive search. He illustrated the 
effectiveness of the algorithm by simulation results. Our paper analyses the speed of the algorithm when a 
certain probability model is assumed. We prove, that if the metric space is the H" with the Euclidean 
metric, then under some suitable conditions, the number of distance calculations required for finding the 
nearest neighbor tends to be negligible compared to that of the exhaustive search as the number of sample 
points tends to infinity. Closely related to the nearest neighbor algorithm we derive a classification 
method with asymptotically the same error probability as that of the nearest neighbor decision, for which 
the number of necessary distance calculations depends only on the dimension of the space rather than on 
the number of the sample points. 


A Parameter Estimation Algorithm for Speech Recognition to Maximize "State Optimized Joint 
Likelihood” 

G. Lugosi and A. Faragd Institute of Communication Electronics, Technical University of Budapest, H- 
IIII Stoczek u2, Budapest, Hungary 

A central problems concerning speech recognition using statistical methods is the estimation of the 
probability distributions of the Hidden Markov Models which arc supposed to describe the time-varying 
properties of the speech signal. The distributions are determined by finite number of parameters and the 
estimation is based on the maximization of an objective function of these parameters. 

Most commonly, the object of the maximization is to obtain maximum likelihood or conditional 
maximum likelihood estimation. Existing algorithms, such as Baum-Welch reestimation or its analog to 
conditional maximum likelihood estimation provide convergence to a local opumum of these objective 
functions. 

State optimized joint likelihood is another objecuve function for which such algorithms exist 
(Jclinek’s "Vitcrbi extraction" or Kabincr, Wilpor, and Juang's "Segmental i-means" algorithm. 

In this paper we present an algorithm which finds the globally optimal parameters-in the sense of 
maximizing state optimized joint likelihood-for the class of lcft-to-right Hidden Markov Models, if an 
observation sequence is given. 


Inferring Optimal Decision Lists from Stochastic Data Using the Minimum Description Length 
Criterion 

Kenji Yamanishi C&C Information Technology Research Laboratories, NEC Corporation, !•!,, 
Miyazaki 4-chome, Miyamae-ku, Kawasaki, 213, Japan 

Inferring classification rules from stochastic data is an important issue in machine learning. This 
paper, in terms of information theory, presents a method for inferring classification rules using Rissanen’s 
Minimum Description Length (MDL) criterion and evaluates the performance of the inferred rales. 

We deal with most specifically the classification rules called "decision lists", which were first 
proposed by Rivest. These specific formulae take the form of linearly sequences of "if - then - else if..." 
type decisions. Although Rivest formulated the decision lists as deterministic rales, we can consider them 
linear sequences of "if - then - (with probability p) else if..." type stochastic decisions, when dealing with 
data from stochastic information sources. Such classification rales define probability distributions of data, 
and each rale itself can be regarded as a probabilistic model of an information source. Inferring decision 
lists is equivalent to estimation of these probabilistic models. 

In this paper, first, we formulate the probability distributions of data with respect to individual 
decision lists Here each decision list has model parameters forming a hierarchical structure. We further 
formulate the prior distributions of those model parameters according to this hierarchical structure 
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Description lengths of data and model parameters are calculated on the basis of these distributions. Next, 
we apply the MDL criterion to selection of the best decision list That is, for any given data, the decision 
list selected as the best will be that which requires the least total description length for the encoding of its 
model parameters and of the data observed through it. Here model parameters are estimated on the basis 
of their hierarchical structure. Finally, the optimality of die inferred decision lists is proved in terms of 
their convergence and the convergence speed with which for any e >0, Prob [DIP* || Pht N )ie) goes to 
zero as N becomes large. Here D(P* || Phtp) is the Kullback-Leibler distance between a true data 
distribution P* and a data distribution Pht N defined by the decision list inferred from i.i.d. N data. Prob is 
N direct product of the probability measure P*. The optimality of the inferred decision lists is also 
experimentally discussed. 


Hidden-Markov-Model Based Optical-Character Recognition • a Novel Approach 

Bor-Shenn Jeng, Fu-Hua Liu, San-Wei Sun, and Tiei-Min Wu Telecommunication Laboratories, 

Ministry of Communications, P.O. Box 71, Chung-U, Taiwan, R.O.C. 

In this paper, we present the application of the Hidden Markov Model as a learning and recognition 
model for midti font Chinese character recognition. This doubly stochastic process encodes the distortion 
and similarity among patterns of a class thru a stochastic and evaluating approach. A simple feature 
extraction is employed in our experiments, which is the histogram of the projected profiles. A character 
class is modeled by a 7-state Hidden Maikov model, and the recognition rate for 1000 multi-font (five 
styles) character classes reaches 98% in average for inside test and 91% in average for outside test. This is 
a novel approach and is rarely applied to resolve character rccogniuon problems in the open literature. 


On-Line Recognition of Handwritten Chinese Characters using Nonlinear String Matching Method 
Chang-Kcng Lin and Bor-Shenn Jeng Telecommunication Laboratories, Directorate General of 
Telecommunications, Ministry of Communications, P.O Box 71, Chung-Li, Taiwan, R O.C. 

This paper proposes an on-line handwritten Chinese character recognition system using a nonlinear 
matching method. The character to be recognized can be stroke-order and stroke-number free, tolerant for 
combined strokes, size flexible, but within the constraint of normal hand-writing. In the first recognizer of 
the algorithm the finite-stale matching mechanism is used to generate a stroke stnng for the input test 
character. In the second recognizer, a windowed-dynamic-programming (WDP) based nonlinear matching 
method is used to perform recognition with the stroke-string features. The alternative linear matching 
method and ns experimental results arc also submitted for comparison and analysis. Reference patterns 
have been generated for 2000 Chinese characters with stroke numbers ranging from 1 to 27. The 
recognition results are based upon the 1800 handwritten characters by 10 people. The obtained 
recognition rate is 94.5%, and the cumulative classification rate of choosing fourth most similar 
characters is up to 99% The multiple dp matching method and subarea matching method implemented in 
the simulation system to achieve rather high speed performance are also presented 


On the Image Recovering from Moment Descriptors 

Miroslaw Pawlak Dept of Electrical Engineering, University of Manitoba, Winnipeg, Manitoba, Canada 
R3T2N2 

The problem of reconstruction of an image from noisy data by the method of moments is examined. 
The set of orthogonal moments based on Legendre polynomials is employed. A general class of signal- 
dependent noise models arc taken into account. 
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An asymptotic expansion for the global reconstruction error is established. This reveals mutual 
relationships between a number of moments, the image smoothness, sampling rate and noise model 
characteristics. 

The problem of an automatic (data-driven) selection of an optimal number of moments is studied. 
This rs accomplished with the help of cross-validation techniques. 
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SESSION FA3 


SIGNAL PROCESSING m 


Performance Analysis of the Constrained LMS Algorithms with Uncorrelated Gaussian Data 
Abraham Krieger Onncon Corporation, 9363 Towne Centre Drive, San Diego, CA 92121 

The convergence properties of a constrained adaptive filtering algorithm are established for 
uncorrelated Gaussian input data. The constraints are in the form of bounded sets in which the filter’s 
coefficients must lie. Two constraint sets in S" are considered: A bounded hypercube with all edges equal 
to B !, and a bounded hypersphere with radius equal to B 2 . A bound on the mean-square-deviation (MSD) 
of the estimate of the filter’s coefficients from its optimal one was obtained in earlier work for dependent 
input processes. This bound is a second-order polynomial of the parameters B„ 1 = 1,2. The goal of this 
work is to show that when the input data is an uncorrelated Gaussian process the bound is a bounded 
funcuon of B, rather than a second-order polynomial. 


A Geometrical Approach to Multiple-Channel Detection 

Douglas Cochran and Herbert Gish Department of Electrical and Computer Engineering, Arizona State 
University, Tempe, AZ 85287, and BBN Systems and Technologies Corporation, 10 Moulton Street, 
Cambridge, MA 02138 

The Generalized Coherence (GC) estimate, a recently introduced detection statistic that forms the 
basis of a non-paramctnc test for common but unknown signal on several noisy channels, is shown to be 
gcomctncal in nature. This perspective is used to show that the GC estimate is the unique function of 
M> 2 complex sample sequences having a desirable set of properties. Its distribution function is derived 
under the H 0 assumption that the sample sequences representing filtered data from M noisy channels are 
drawn from independent Gaussian processes. This is accomplished by exploiting the interpretation of the 
MSC esumate as the volume of a polytope in (real) 2Af-dimensional space. The properties and 
performance of the GC estimate as a dctccUon staustic are also discussed. Detection thresholds are 
derived corresponding to different false alarm probabihucs for various numbers of channels and sequence 
lengths. The performance of the GC detector as a function of the signal-to-noisc ratios on the noisy 
channels is evaluated by simulauon and compared to performance data for other multiple-channel 
detection schemes. 


A Fast and Effective Algorithm for Sinusoidal Frequency Estimation 

S. F. Hsich, K. J R. Liu, and K. Yao Electrical Engineering Department, University of California, Los 
Angeles, Los Angeles, CA 90024-1594 

Recently, a mynad of researches have been focused on SVD and cigcn-dccomposition analysis of 
multiple sinusoidal frequency estimations, which require itcrauve massive computations, hence are quite 
intractable for real time applications Here we propose a dtrcct method, involving only a fixed number of 
operations, called truncated QftiTQR), which is a simplified modification of a SVD-based FBLP method 
developed by Tufts and Kumarcsan. Since the QR method is amenable to VLSI (e.g., systolic array) 
implementauon, a rcal-umc processing for idenufying multi-harmonics is thus feasible under stauonary 
and time-varying conditions. 

A FBLP model is used in the solution of the rank-reduced LS problem. Then an AR pseudo- 
spectrum is plotted, where the peaks yield the locations of the harmonics embedded in the noise perturbed 
data samples observed over a finite duration. Our method shares the benefits of identifying narrowly 
spaced harmonics as well as suppressing spurious frequency peaks such as in the truncated SVD(TSVD) 
approach. However, the computational cost has been greatly reduced. Computer simulations are provided 
for comparisons to other known approaches and show that the TQR method is a promising candidate for 
the efficient retrieval of clustered harmonics in real time applications 
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Time-Frequency Filtering and Synthesis from Convex Projections 

Langford B. White Electronics Research Laboratory, Defence Science & Technology Organization, P.O. 
Box 1600, Salisbury, SA„ 5108 Australia 

This paper describes the application of the theory of projections onto convex sets (POCS) to time- 
frequency filtering and synthesis problems. We show that l s class of Wigner-Ville Distributions (WVD) 
of Li signals form the boundary of a closed convex subset of LiOR 1 ). This result is obtained by 
considering the convex set of states on the Heisenberg group, of which the ambiguity functions (AF) form 
the extreme points. The form of the projection onto the set of WVD's is deduced. Various linear and 
nonlinear filtering operations are incorporated by formulation as convex projections. An example 
algorithm for simultaneous time-frequency filtering and synthesis is suggested. 


Modifying Real Convolutional Codes for Protecting Digital Filtering Systems 
Robert Redinbo and Bernhard Zagar Dept, of Electrical Engineering and Computer Science, University 
of California, Davis, Davis, CA 95616, and Institut fiir Allgemetner Elektrotechmk und Elektrtsche 
Messtechnik, Technical University of Graz, A-8010, Graz, Austria 

Digital filters when implemented with very dense high-speed electronic devices are susceptible to 
both temporary and permanent failures, not easily protected by conventional fault-tolerant computer 
design principles. A new method is presented for protecting the overall realization against both hard and 
soft errors at the data sample level using the error-detecting properties of real convolutional codes The 
normal filter system is surrounded with parallel parity channels defined by a real systematic convolutional 
code. Erroneous behavior is detected by comparing externally the calculated and regenerated pamy 

samples. A rate — real convolunonal code produces n-k) parity samples for every k input samples 

R 

causing the parity channels to operate at a rate decimated by k. Significant complexity reductions are 
possible by modifying the code structure, without loss of error protection, yielding simplified parity 
channels with finite impulse response (FIR) structures operating at rate decimated by k. The theory and 
practice of the code modification procedure depend on an annihilator subspace associated with the 
columns of a code segment matnx This subspacc is affiliated with the dual code description. Among 
other things, the parity channels are redesigned to cancel poles associated with the onginal transfer 
function. A set of homogeneous equations determine the class of solutions by interrelating dependent and 
independent choices for the modified pamy channels’ FIR parameters 


First Order Error PDF for Nonlinear Stochastic Filters 

Carlos A. C. Bclo and Josd M. F Moura Dept, of Elect, and Comp. Eng , Carnegie Mellon University, 
Pittsburgh, PA 15213 

The paper studies the characterization of the error for nonlinear estimation problems. In these 
problems, filters are designed to provide estimates of an underlying unobserved process (the message of 
interest) from noisy measurements of a nonlinear function of the message. An outstanding unresolved 
question is the performance evaluation of the optimal as well as suboplimal nonlinear filters. In this work, 
we consider directly the first order characterizauon of the error process p (f.) of any nonlinear filter F. It 
turns out that this function is obtained as an expectation of a translated version of the a posteriori pdf. 

Let x be the signal to be estimated and Z the available observations. If p v (Z) = p, | Z (X |Z) is the 
conditional density of x given Z then the filter F 1 that computes the estimate x =F '(Z), exhibits an 
error e =x-x with probability density function p f c ' (X) = E[p ap (X + x')] = E[pc\z(X)l where £[•] is 
the expectation operator taken over the underlying joint probability space. 

The technique has been applied to the absolute phase demodulation problem. The message is a 
two-dimensional signal that phase modulates a carrier. The error probability function has been computed 
for several nonlinear phase estimators using the method desenbed above. Movies of the error pdf’s for 
these filters will be presented. 
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Fast "Modified Triangular Factorization" of Hermitian Toeplitz and Quasi-Toepfitz Matrices with 
Arbitrary Rank Profile 

Debajyoti Pal and Thomas Kailath Information Systems Laboratory, Stafford University, Stanford, CA 
94305 

We present a fast procedure for computing a “modi fled triangular factorization” of certain structured 
matrices in 0(/t 2 ) operations. This family of structured Hermitian matrices includes Toeplitz and Quasi- 
Toeplitz (matrices with certain hidden Toeplitz structure). 

The derivation of this procedure is based upon a function-theoretic interpretation of the so-called 
“displacement structure" exhibited by these matrices. This interpretation has been used by Lev-Ari and 
Kailath to derive fast triangular factorization procedures for the “strongly regular” Hermitian matrices 
with a so-called "generalized displacement structure.” In this paper we have modified their procedure for 
"strongly regular” Toeplitz and Quasi-Toeplitz matrices to handle matrices with the same structures but 
having arbitrary rank profile. In case of “strong regularity" our new procedure produces the same result as 
Lev-Ari and Kailath's. If "strong regularity" is lacking then our procedure recursively produces a LDL* 
factorization where D is a block diagonal matrix. Rank profile and inertia are also computed with no extra 
effort 


An Order Selection Rule fur Rank Reduction in the Linear Statistical Model 

Richard T. Behrens and Louis L. Scharf Department of Electrical & Computer Engineering, University 

of Colorado, Boulder, CO, 80309 

A new order selection rule is proposed for rank reducuon in the linear statistical model. Rank 
reduction decreases noise-induced estimator variance at the expense of increased estimator bias, but 
results in lower mean squared error. The new order selection rule is to find the maximum likelihood 
estimate of the mean squared error between the true signal and a reduced rank signal estimate, and choose 
the rank for which this esumated mean squared error is smallest. 
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SESSION FA4 


CODING THEORY Vm 


Extremal Codes are Homogeneous 

Vera Pless Mathematics Department, University of Illinois at Chicago, Chicago, Illinois, 60680 

We show that extremal codes are homogeneous. This implies that each punctured code of an 
extremal code has the same weight distnbution, which can be calculated directly from the weight 
distribution of the parent extremal code. 


Extremal Codes of Length 40 and Automorphism of Order 5 

V. Y. Yotgov and N. P. Ziapkov Mathematics Department, Higher Pedagogical Institute, Shumen 9712, 
Bulgaria 

Doubly-even self-dual (DESD) codes exist only in lengths n that are multiple of 8. All such codes 
are completely determined up to length 32. Those of DESD codes that have the greatest possible weight 
4[n/24j +4 arc called extremal. Many extremal DESD codes of length 40 have been constructed recently 
via Jadamaid matnccs, some of them having a very small or tnvial automorphism group It is known that 
the only primes that can divide the group order of an extremal DESD code of length 40 are 19, 7, 5, 3, 
and 2. All such codes having automorphism of order 19 or 7 are known. Using the well-developed 
techniques for constructing self-dual codes via automorphisms, we obtain a complete list of all extremal 
DESD codes of length 40 having automorphism of order 5 


Equivalences of Binary Irreducible Goppa Codes 

Hermann I. Helgcrt Department of Electrical Engineering and Computer Science, The George 
Washington University, Washington, DC 20052 

We consider the class of binary irreducible Goppa codes of length 2 T generated by the elements of 
GF(2') and the Goppa polynomial g(z) =g 1 (z) 2 ', where gj(r) is a polynomial of degree p with 
coefficients from GE(2 I ) that is irreducibie over GF( 2 1 ) and t is a positive integer. It is known that 
among this class there exist codes that asymptotically satisfy the Varshamov-Gilbert bound on 
performance as r becomes large, although the problem of analytically determining which g(z) generates 
such codes has not been solved It is therefore of interest to develop methods for reducing the scope and 
complexity of a computer-based evaluation of the weight spectra of these codes. Toward this end we 
show that certain linear transformations of the roots of g i (r) yield Goppa polynomials that generate codes 
with identical weight spectra. This property allows us to collect the set of all irreducible Goppa codes of 
given degree into equivalence classes, thereby considerably reducing the number of codes that must be 
evaluated. We also obtain a result that relates the size of an equivalence class to the number of conjugates 
of a certain invariant of the roots of gi(z) and derive upper and lower bounds on the number of 
equivalence classes for given r,/t and t that are asymptotically ught as t becomes large. 


The Automorphism Group of the Kerdock Code 

Claude Carlet Umversite'Paris VI, Laboratoirc cTInformatique The'orique et Programmation (UTP), 4 
place Jussieu, 75252 Paris Cedex 05, France 

We prove that the automorphism group of the Kcrdock code of length 2™, m even and m >6 is the 
group of the following permutations on GF(2 m ~')xGF(2) % 

(x,e)e GF(2 m ' l )xGF(2): (ax 2 ' + b, e + S) 

where a ranges over G/ r (2'"' 1 )-(0), b ranges ovcrG/ r (2"' 1 ), 5e GF( 2) and is [0, m-2}. 
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Thus, the Kerdock and Pieparata codes of same length, which are known to be formally dual, have 
also the same automorphism group (W. Kantor has determined Preparata's one). 

In the proof of this new result, we solve the principal problem by using the relation between the 
weight of an element of the Reed-Muller code of order 2 and the rank of its symplectic form. We use also 
and prove the following lemma: 

Let 'f be an automorphism of the linear space G' = GF (2 m_1 ), and /be a linear form on G‘ such that 
for all (x,y) e G' 2 we have: 

( 4-(x) H'(y)) 2 "’ +f(x)'¥(y)+fly) 'P(x) +4'((xy) 2 “‘) = 0. 

Then/is the null function and ¥ is of the type: 

x-*ax 2 , ie[0,m-2], aeG'-fO) 


There is No (24,12,10) Self-Dual Quaternary Code 

Clement Lam and Vera Pless Department of Computer Sciences, Concordia University, Montreal, 
Quebec, Canada H3G IM8, and Department of Mathematics, Statistics and Computer Science, 
University of llinois at Chicago, Chicago, IL 60680 

The question of the existence of a (24,12,10) self-dual quaternary code has been a long-standing 
one. The largest minimum weight a self-dual quaternary ln,nl 2) code can have is d = 2(n/6)+2 and a 
self-dual quaternary code attaining this minimal weight is called an extremal code. Extremal codes exist 
of all even lengths n less than or equal to 22. If an extremal quaternary code of length n = 24 exists, 
vectors of weights 10,12 and 14 would hold 5-designs by the Assmus-Mattson theorem. We approached 
this problem by attempting to construct the generator matrix of this code directly. The conclusion we 
reached is that there is no such code. 


The Carlitz-Uchiyama Bound and the Dual of the Melas Code 

Gilles Lachaud Equipe "Arithmetique et The’one de U Information"CIRM., Lummy Case 916, 
FI3288-Marseille Cedex 9, France 

The first part is devoted to the Carlitz-Uchiyama bound, which gives estimates on the weights of the 
words of the duals of BCH codes in characteristic two; as is well-known, its proof lies on some kind of 
Weil's inequality on exponential sums. We give here a bound for the duals of geometric BCH codes on 
curves of arbitrary genus on any prune field, i e., for the subcodes over the prime field of geometric 
Goppa codes. To this end we give a sharpened bound for a quite general family of exponential sums along 
a curve. We thus give also some bounds on the number of solutions of the equation Tr If (x) = 0, where / 
is a function define on a curve. 

The second part is concerned with the study of a particular code in characteristic two, namely the 
dual of the Melas code that we call the Kloosterman code. The weights are connected to the classical 
Kloosterman sums. For this code we compute the weights exactly occurring, the multiplicity of weights 
in terms of class number of quadratic forms, and we show that in some sense these weights become 
cquidistnbutcd for a particular measure when q is large. 


Exponential Sums and Goppa Codes 

Carlos J Moreno and Oscar Moreno Baruch College and Graduate Center, CUNY, 17 Lexington Ave., 
Box 509, New York, NY 10010, and Department of Mathematics, University of Puerto Rico, Rio Piedras, 
Puerto Rico 00931 (40 min.) 

Recently we obtained a sharpening of the Cariitz Uchiyam bound in charactcnstic two using Serre's 
improvements on Weil's esumate of the number of rational points on an algebraic curve over a finite 
field, in this talk we denve an improvement, also on characteristic two, on a theorem of Bombicn and 
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Weil, this has important applications in the theory of binary codes; for instance, we obtain immediate 
results concerning the minimum distance of the dual of an arbitrary subfield subcode of a Geometric 
Goppa. Furthermore, we estimate the covering radius, and we find the exact value for the minimum 
distance and the number of information symbols whenever die minimum distance is small in relation to 
the length of the code. 

Research problem 12.1 in Mac Williams & Sloane "The Theory of Error Correcting Codes" asks for 
the hue dimension and minimum distance of a classical Goppa Code. Using the above results for 
Geometric Goppa codes in the genus 0 case, we can solve the problem for the minimum distance of 
classical Goppa codes whenever die length of the code n = 2 m satisfies a certain inequality on the degree 
of the Goppa polynomial. On the other hand, given that the approach in the first paragraph makes use of 
heavy number theory arguments, we will also derive an elementary proof of the improvement on the 
conditions of the theorem of E. Bombieri, which will give us a valid proof of the result in the case of 
classical Goppa codes. We use this improvement also to generalize a previous result on the minimum 
distance of the dual of a Goppa code. 
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SESSION FAS 


ENTROPY 


Entropy and Surface Entropy of Random Fields on Trees 

Toby Beiger and Zhongxing Ye School of Electrical Engineering and Center for Applied Mathematics, 
Cornell University, Ithaca, NY 14853 

We study entropic aspects of several models of random fields on trees. For any shift invariant 
random field on a open Cayley tree, the entropy rate and surface entropy rate exist. In particular, for a 
Maikov i bain field the entropy rate has a simple form similar to that of a one-dimensional Markov chain. 
However, surface entropy rate and conditional surface entropy rate, which are trivial in the one¬ 
dimensional case, exhibit interesting behaviors when extended to Maikov fields on trees. The above 
results are extended to any open Bethe tree on which each vertex has the same number of neighbors. We 
also consider random fields defined on closed trees with recursive structure. It is shown that the entropy 
rate exists for such random fields if they possess appropriate stationary properties. 


The Entropy Power and Related Inequalities 

Amir Dembo Information Systems Laboratory, Stanford University, Stanford, CA 94305 

Two proofs of t he En tropy Power Inequality (EPI) are presented. It follows from the convexity 
inequality h (yXX + Vl -A Y) 2 Xh (X)+ (1 -X)h (Y), for independent random variables X, Y and 
OS A 5 1. Stam’s proof of the EPI via implicit normal perturbations is simplified as explicit normal 
perturbations prove the convexity inequality. 

A new proof of the convexity inequality via a limiting argument on Bcckner’s sharp version of 
Young's inequality is presented A similar limiting argument on the converse of this inequality yields an 
analogue convexity inequality about Minkowski-sum of sets, which in turn results in the Bnrnn- 
Minkowski Inequality (BMI) Bcckncr’s version of Young’s inequality, and its converse, are restated in 
terms of Rcnyyi differential entropies, thus filling the gap between the BMI and EPI 

In addition to the isopcnmctric inequalities that result from the EPI and BMI, a third isopenmetric 
inequality is denved from an inequality about Fisher informations This isopenmetric inequality amounts 
to Costa's recent result on the concavity of the entropy power, thus supplying a short and elegant proof of 
the latter 

Graph Entropy and Convex Programming Dualities 
Victor K. Wei Bellcore, 435 South Street, Morristown, NJ 07960 

Shannon defined the (zero-error) capacity of a graph to be the maximum reliable transmission rate 
of a channel corresponding to the graph Lovasz in 1979 derived the capacities of the pentagon and other 
graphs. (Conner generalized the concept to include probabilistic considerations and defined the entropy of 
a graph. Recently, Csiszar, (Comer, Lovasz, Marton, and Simonyi obtained a characterization of 
anublocking pairs of polyhedra and of perfect graphs based on a certain strong splitting property of graph 
entropy. 

Motivated by a resource allocation problem, Wei, and Monma, Schrijver, Todd, Wei studied a 
convex programming problem over acyclic directed graphs. Their results include vanous dualities of 
convex programs, approximate solution algonthms, and a generalization of the LYM property for 
partially ordered sets. 

Here, we show that these two seemingly unrelated scries of results can be unified. The entropy 
splitting property actually corresponds to a strong convex programming duality. With this approach, we 
obtain a simplified proof of the results in Csiszar, (Comer, Lovasz, Marton, and Simonyi and gcncraliza- 
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tions. This may lead to a better understanding of the Shannon capacity of graphs and of several multi¬ 
user information theory problems. 


Tight Upper Bounds on the Entropy Series 

Renato M. Capocelli and Alfredo De Santis Dipardmento di Matematica, Vniversitd di Roma, 00185 
Roma, Italy, and IBM T J. Watson Research Center, P.0 Box 218, Yorktown Heights, New York, 10598 
Two tight upper bounds on the entropy H(P) of a countable random variable are provided, the first 
bound, expressed in terms of £ p (log), is 

H(P)ZpE p (lo$)+\og((ti), 

where f(/t) is the Riemann zeta function, ji is the unique solution of the equation 
E p (log) = and &(k) is defined as In p when it is a power of a prime p; 0 otherwise. The 

bound improves previous known upper bounds. 

The second bound, expressed m terms of E/LlogJ), is 

H(P) < £„(LlogJ)+ (£,(LlogJ +1)H 


1 

£„(LlogJ)+l 


where H (p) is the Shannon function -p log p -(1 -p)iog(l -p). 

This latter bound allows to denve an easily computed approximation of the former and a bounded 
error estimate of it (the exact estimate would involve a complex evaluation of the Riemann zeta function) 


It £,(log)+logfOt) = £)(log) 


1 + H( 


£,0og) 


+ a. 


where a is a bounded function of £,,(log), i.e., -1 < a S A p -\,A p S 3 if lS£ ? (log)<p and 
A, < 1 + H (l/£,,(log)), otherwise; and tp is the golden ratio (l + v5)/2. (This work was partially 
supported by the Italian Ministry of Education, Project: Progetto ed Analisi di Algontmi) 


When is Graph Entropy Additive? Or: Perfect Couples of Graphs 

Jdhos Komcr, Gdbor Simonyi, and Zsolt Tuza Mathematical Institute of the Hungarian Academy of 
Sciences, H-1364 Budapest, FOB 127, Hungary, and Computer and Automation Institute of the 
Hungarian Academy of Sciences, H-Illl Budapest, Kende u. 13-17, Hungary 

Graph entropy is an information theoretic functional on a graph and a probability distnbution on its 
vertex set. fixing the probability distnbution, it is sub-additive with respect to graph union. This property 
has been used to derive non-existence bounds in graph covcnng problems by several authors. 

Here we are dealing with the conditions for additivity instead of sub-additivity. For two 
complementary graphs it was proved by Csiszdr el al. that additivity for every possible probability 
distnbution is equivalent to the perfectness of the involved graph(s). here we give necessary and sufficient 
conditions of additivity (for every prob. dist.) in full generality. This can be considered as a generalization 
of the concept of perfect graphs, giving some insight into the kind of graph properties to which graph 
entropy is sensitive. 

Maximum Growth Exponent Equals Minimum Information Rate 
Paul Algoet I/formation Systems Lab, Durand 141B, Stanford, CA 94305 

We formulate a convex duality principle and prove that the maximum growth exponent, taken over 
a convex family of functions, is equal to the minimum informauon rate, taken over the polar family of 
measures. This minimax property remains valid when sequences are selected from convex families so as 
to maximize the asymptouc growth exponent. The results are related to the variational characterizations 
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of relative entropy rate that is due to Donsker and Varadhan, and to the asymptotic optimality principle 
for log-optimum investment and gambling that was formulated by Algoet and Cover. 

Moving Average Processes and Maximum Entropy 

Dimitris Nicolas Politis Department of Statistics, Staiford University, Stanford, CA 94305-4020 

Let X, % te Z be a wide sense stationary stochastic process with mean EX, = 0 and autocovariance 
y(k) = EX,X, +k . tfce Z. It is well known tBurg 1967] that the Maximum Entropy such process that satisfies 
the constraints yO') = c,, i = 0,1 ,...,p is the AR (autoregressive) Gaussian process that satisfies these 
constraints. Physical or practical considerations might in some cases impose the additional constraint that 
y(i) = 0, i > q. Since any time series with y(i) = 0, i > q, is a MA (moving average) process of order (at 
most) q, we then face the problem of finding the Maximum Entropy process among the MA (?) processes 

that satisfy the constraints y(i) = c, i = 0. p. The solution to this problem rests upon the relationship 

between the autocorrelation and inverse autocorrelation function of an AR process that was recently 
brought to light by Kanto (1987). It is to be noted that in the context of spectral estimation, q=p 
corresponds to a periodogram-like estimator, while q = °° leads to Burg's all-pole (AR) estimator. Hence 
the choice p <q <°° yields a solution intermediate between the penodogram and Burg’s AR estimator. 


Some Correlation Properties of and Entropy Calculations in 2-D Lattice Filters 
A. Ertuzun and E. Panayirci Department of Electrical and Electronic Engineering Bogazici Universitesi, 
Bebek, Istanbul, Turkey, and Faculty of Electrical and Electronics Enginee. ing, Ayazaga Kampusu, 
Istanbul, Turkey 

The theory of one-dimensional (1-D) latuce filters are well developed and well knowa In recent 
years, two dimensional (2-D) lattice filters are being investigated intensively and their theoncs are being 
developed. 2-D latuce filters with quarter plane support have many similarities with the causal 1-D lattice 
filters Making use of these similantics, the correlauon relauons between the forward and the backward 
prcdicuon errors arc derived. However, there are some differences between the 2-D and the 1-D lattice 
filters due to the differences in the growth of data support Even though the data support of a 1-D lattice 
filter grows linearly with the order of the filter, the growth is not linear in a 2-D case since the support 
grows in both directions. Thus a latuce filter with fixed number of coefficients is not sufficient to model a 
2-D autoregressive (AR) data field The information is lost while modeling a data field with fixed number 
of reflecuon coefficients. An expression is derived to express the autocorrelation matrix of the backward 
prediction errors in terms of the reflection coefficients and the residual error power at each stage The 
entropy of the backward prediction errors is calculated in terms of its autocorrelation matnx and it is 
compared with that of the data field. The entropy lost is calculated quantitatively by computer simulafions 
and the theory is confirmed. 
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Multilevel Codes with Bounded Af-th Order Running Digital Sum 

E. Eleftheriou and R. Cideciyan IBM Research Division, Zurich Research Laboratory, CH-8803 
Riischlikon, Switzerland 

Multilevel sequences with a spectral null of order M, that is, the power spectral density and its first 
2 M -1 derivatives vanish, are characterized by finite-state transition diagrams, whose edge labels satisfy 
bounds on the variation of the Af-th order running digital sum. Necessary and sufficient conditions for 
sequences exhibiting a spectral null or order M are given. For this new class of codes a lower bound on 
the minimum Euclidean distance at the output of partial response channels with spectral null of order P is 
obtained. It is shown that the distance bound depends on the sum of the orders of code and channel 
spectral nulls and can be met with equality provided that M + P < 10. The case of M + P > 10 leads to an 
unsolved problem in number theory. Examples of quaternary codes on dicode channels and their 
maxentropic power spectral density arc given. 


Parallel and Variable Coding/Decoding of MDS-Codes 

B. G. Dorsch Technische Hochschule Darmstadt, Institut fur Netzwerk- und Signatheorie, Mcrckstrasse 
25, D-6100 Darmstadt, FJR. Germany 

Usual coding/decoding algorithms for Maximum-Distance-Separable (MDS)-Codes, as Reed- 
Solomon (RS)- or extended RS-codes, either are not variable in codeparameters (n,k), n = codelength, k - 
dimension, and/or don’t have a parallel structure as required for fast processing. New algorithms with 
variable and parallel structure based on Newton’s inteipolation of polynomials are described, suitable for 
general purpose RS-codcrs/decodcrs (over a fixed field GF(q)), including simplifications for correction of 
errors and erasures. The corresponding syndromes can be used for arbitrarily shortened, extended 
permutated or punctured RS-codcs either with the Berlekamp-Masscy-Algorithm (BMA) or the 
Euclidian-Division Algorithm (EDA) to calculate error-location- and error-value-polynomial. Simple 
relations to the usual Discrctc-Fouricr-Transform (DFT)-rcprescntation of MDS-codes are given. Further 
simplifications of coding/decoding for codes over the field of real or complex number (for ’coded 
modulation’) are indicated. 


A New Table of Constant Weight Coder 

A. E. Brouwer, James B. Shearer, N. J. A. Sloane, and Warren D. Smith Technological University of 
Eindhoven, 5600 MB Eindhoven, Netherlands; IBM Thomas J. Watson Research Center, Yorktown 
Heights, NY 10598; and AT&T Bell Laboratories, Murray Hill, NJ 07974. 

This paper presents a table of binary constant weight codes of length n < 28. Explicit constructions 
are given for almost all the 600 codes in the tabic; most of these codes arc new. We briefly survey the 
known techniques for constructing constant weight codes, and also give a table of (unrestricted) binary 
codes of length n <28. 


The Nonexistence of t-QJP Codes, for t > 2, and Some New 2-QP Codes 

Behnam Kamali and Harold Longbotham Division of Engineering, University of Texas at San Antonio, 
San Antonio, TX 78285 

Quasi-perfect (QP) codes are optimum for the binary symmetric channel in the absence of perfect 
codes. We prove that there are no linear t-QP codes for t > 2, where t is the number of errors the code can 
correct. This proof is based on the properties of parity check matrices for linear block codes. A new 




search method for 2-QP codes is described. Several new 2-QP codes, found using this method, are 
reported. 


Construction of Optimal or Nearly Optimal m-Out-Of-n Codes Through Arithmetic Coding 
Tenkasi V. Ramabadran Department of Electrical and Computer Engineering, Iowa State University, 
Ames, Iowa 50011 

A ( n,k ) block code in which each binary codeword of length n has exactly m Is is referred to as a m- 
out-of-fi code. Such codes are useful, for example, in providing perfect error detection capability in 
asymmetric channels such as optical communication links and laser disks. A m-out-of-n code is optimal 
when, for a given number of information bits k, and m, the number of check bits r (= n-k) is minimum 
(r opl ). The code is nearly optimal when r = r opt +1. In this presentation, we describe a new scheme for the 
construction of m-out-of-n codes based on the arithmetic coding technique. Basically, arithmetic 
decompression is used to encode a /fc-bit information word into a n-bit codeword and arithmetic 
compression is used for the decoding operation. The encoding and decoding algorithms of the scheme 
involve only simple arithmetic operations performed recursively, thereby facilitating the construction of 
codes with relatively long block sizes. The scheme allows the construction of optimal or nearly optimal 
m-out-of-/i codes for a wide range of block sizes limited only by the arithmetic precision used. 


Anticode Construction and Bounds on Maximum Distance 

Valdcmar Cardoso da Rocha, Jr., and Marcia Mahon Campello de Souza Department of Electronics and 
Systems, Communications Research Group - CODEC, Federal University of Pernambuco, Cidade 
Universitaria, 50.741 Recife PE, Brazil 

Two techniques are presented for constructing new anticodes from known anticodes, namely 
product and interleaving of anticodes. The product of two anlicodcs with parameters (m i,* t 5i) and 
(m 2i k^ S 2 ) respectively, produces an ( m,k,S) anticode, where m = m\m 2t k = k\k 2 and 
<5<min[mi 8 2< m 2 S\). It is also shown that an interleaving of degree X of an ( m,k , 8) anticode produces 
an (mX,kX,8X) anticode. Many of the anticodes constructed are either optimum or near optimum in the 
sense of having the lowest maximum distance 5 for given values of m and k. Another topic covered in 
this paper is relative to bounds on the maximum distance 8 of an anticode. A Gilbert-Varsharmov type of 
upper bound and a tighter form of the Plotkin lower bound on the maximum distance of linear anticodes 
are presented. 


Nonsystematic d-Unidirectional Error Detecting Codes 

Eiji Fujiwara and Masayuki Sakura Dept, of Computer Science, Tokyo Institute of Technology, 
Ookayama, Meguro-Ku, Tokyo 152, Japan 

This paper presents a simple construction method for the optimal nonsystematic d-unidirectional 
error detecting (d-UEC) codes. A necessary and sufficient condition for the d-UED codes, C, is stated 
such that for all X, Y e C cither X and Y are unordercd or the Hamming distance between them is at least 
d + 1 when one covers the other. First we adopt the weight zero codeword with length n and then adopt 
weight d + 1 constant-weight codewords. Next, codewords with weight 2(d + 1), 3(d + 2), • • • , arc 
adopted until the weight is less than or equal to the codeword length, n. In this case, it can be easily found 
that the number of codewords can be varied by adopting the nonzero codewords as the first step in the 
above construction method. We conclude that adopting weight |[n/2j |^+i codewords as the initial set of 
codewords in the above method and then adding codewords with weight d + 1 to the previous ones gives 
the maximum number of codewords of the d-UED codes, which is equal to the theoretical bounds. (|A | y 
means the residue of A' divided by y, and |*J means the maximum integer no greater than x.) 




174 




A New Class of Constructive Asymptotically Good Generalized Concatenated Codes Beyond the 
Zyablov Bound 

Toshihisa Nishijima, Hiroaki Ishii, Hiroshige Inazumi, and Shigeichi Hirasawa Faculty of Engineering, 
Kanagawc Institute of Technology, Kanagawa, 243-02 Japan, Sagami Institute of Technology, 
Kanagawa, 251 Japan, and School of Science and Engineering, Waseda University, Tokyo 169, Japan 

A new class of constructive asymptotically good codes are proposed by using the construction 
method of the generalized concatenated codes. The outer code of the codes is formed from J(J>2) 
Reed-Solomon codes with the same code length and the different code rates. The inner code of the codes 
is the interleaved code constructed by the base code with known weight distribution, proposed by E. J. 
Weldon, Jr., where the inner code is a nonsystematic code and has 7 -1 subcodes. The lower bounds for 
the new codes are the best bounds for code rate r, 0.0741 <r <0.344, in all the constructive 
asymptotically good codes previously known to us. Moreover, we obtain an interesting result that those 
for the new codes lies more above the Zyablov bound for code rate, 0.0838 <r < 0.354. 
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CONVOLUTIONAL CODES 


On Bit-Error Probability for Convolutional Codes 

Marat V. Bumashev and David L. Cohn Institute for Problems in Information Transmission, USSR 
Academy of Sciences, Ermolovoy str. 19, 101447, Moscow, and Department of Electrical & Computer 
Engineering, University of Notre Dame, Notre Dame, IN 46556 (40 min.) 

A new analytic expression for bit-error probability for convolutional codes is proposed. The 
expression uses the new concept of basic metric state and the associated notion of a new time scale. 
These allow the behavior of a decoder of convolutional codes to be viewed, to a certain extent, like a 
decoder of block codes. The expression is derived for performance over the binary symmetric 
memoryless channel by rate 1 In codes. It is used to find improved upper bounds on bit-error probability 
and to analyze its asymptotic behavior. 


A Class of Self-Orthogonal Convolutional Codes 

Valdemar C. da Rocha, Jr. Department of Electronics and Systems, Communications Research Group - 
CODEC, Federal University of Pernambuco, Cidade Universitaria 50.741, Recife PE, Brazil 

This paper presents a systematic procedure, based on linear congruences, for the constniction of a 
class of binary self-orthogonal convolutional ( n,k,m ) codes of minimum distance d=n-k+ 1 =/+ 1, 
where k = m + 1 =p, 1 <J <p, and p is a prime number. 


Ring Convolutional Codes for Phase Modulation 

James L. Massey, Thomas Mittclholzcr, Thomas Riedel, and Mark Vollenweider Inst, for Signal and 
Info. Proc., Swiss Federal Institute of Technology, 8092 Zurich, Switzerland 

It is argued that "linear codes" over the ring of integers modulo M are the natural linear codes for 
use with phase modulation. A ring weight and a ring distance can be defined such that the ring distance 
between two codewords equals the ring weight of their difference and also equals the squared Euclidean 
distance between the corresponding sequences of modulated signals. Moreover, a phase shift of 360 /M 
degrees in a modulated signal corresponds to adding 1 to the associated ring clement. A list is given of 
ring convolutional codes for 4-phase and 8-phase modulation that have generally at least as large free 
Euclidean distance as the best trellis codes with the same number of states found by applying subset 
partitioning to ordinary convolution codes but arc, in contrast to the latter codes, invariant to a phase shift 
of 360 /M degrees. 

An appropriate theory of M- ary ( n,k ) ring convolutional codes is developed. The code is defined as 
a rank n free module of /i-tuplcs with entries in the ring of M- ary casual rational functions. When M is 
the e-th power of a prime p, the set of code sequences for an ( n,k ) ring code coincides with that of an 
ordinary ( ne,ke ) convolutional code over GF(p)\ a linear encoder for the ring code is catastrophic and/or 
minimal according as its field code counterpart is catastrophic and/or minimal. A simpler test for 
catastrophity of the ring encoder is given. It is shown that (unlike in the field case) a ring convolutional 
code can have no non-catastrophic polynomial encoder and can have no minimal polynomial encoder, but 
(like in the field case) always has a non-catastrophic and minimal systematic encoder. 
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A Convolutional Decoding Structure for High Data Rate Applications 

R. Schweikert and A. J. Vinck German Aerospace Research Establishment (DLR), Institute for 
Communication Technology, Oberpfaffenhofen, D-8031 Wessling, Federal Republic of Germany, and 
Eindhoven University of Technology, Information-Communication Theory, P.O. Box 513, NL-5600 MB 
Eindhoven, The Netherlands 

We describe a concatenated coding scheme suitable for high data rate applications. The structure of 
the considered scheme basically consists of a set of parallel operating short constraint length K = 3 
convolutional codes as inner code and a single parity check outer code. Consequently the first step in the 
decoding is performed by a set of parallel operating 4-states Viterbi-decoders. The outputs of these 
decoders are used as input for the soft-decision single parity check outer decoding. By using the analogue 
demodulator outputs the outer decoder is able to correct one erroneous inner Viterbi-decoder output. The 
embodiment of the calculations to facilitate this operation is shown to be of low additional complexity. 

The structure of the resulting decoder is characterized by its high parallelism. The overall hardware 
to implement the decoder is roughly the same as that of a K -1, 64-states Viterbi-decoder. However, 
there is a significant gain in decoding speed. Furthermore the error event length distribution is basically 
determined by the event length distribution of a K = 3 Viterbi-decoder. 

The derived upper-bound for the decoding bit-error-rate shows that approximately the same coding 
gain as for the well-known constraint length K = 7 convolutional code with regular soft-decision 64-states 
Viterbi-decoding can be expected. 


An Upper Bound on the Error Performance of Convolutional Coding with Nonindependent 
Rayleigh Fading 

Francois Gagnon and David Haccoun Department of Electrical Engineering, Ecole Polytechnique de 
Montreal, Campus de /’ Universite'de Montreal, P.O. Box 6079, Succ. "A", Montreal (Quebec) H3C 3A7 

A new upper bound is proposed for the evaluation of the error performance of coded systems over 
noninterleaved or partially interleaved Rayleigh fading channels. The correlation between successive 
received symbols is exploited to bound error performance. The bound allows useful evaluation of coding 
gains on realistic communication systems without going into lengthy computer simulations. It may also 
provide a useful tool for the search of good codes on continuous channels with memory. 


Bidirectional Algorithms for the Decoding of Convolutional Codes 

David Haccoun and Jean Belzile Department of Electrical Engineering, Ecole Polytechnique de 
Montreal, P.O. Box 60-79, station "A", Montreal (Quebec) Canada H3C 3A7 

New bidirectional multiple-path tree searching algorithms for the decoding of convolutional codes 
arc presented. These suboptimal decoding algorithms, suited for long constraint length convolutional 
codes, use several paths, all of equal length, in a bidirectional breath-first tree searching manner. It is 
shown that this bidirectional exploration alleviates some of the bit error propagation due to the correct 
path loss of the usual Af-algorithm, Computer simulations with M - 32, 64, and 128, show that a coding 
gain of 1 dB at Ps = 10" 5 can be achieved over Vitcrbi decoding using the same number of paths. 


Some Easily Analyzable Convolutional Codes 

Sam Dolinar, Robert McElicce, Fabrizio Pollara, and Henk van Tilborg Department of Electrical 
Engineering and Jet Propulsion Lab, California Institute of Technology, Pasadena, CA 91125, and 
Department of Math and Computing Science at Eindhoven University of Technology, Eindhoven, The 
Netherlands 

In this paper we will study a class of convolutional codes which we call zero-run length (ZRL) 
convolutional codes. An ( n , 1) convolutional code of constraint length K is said to be ZRL if the 
encoder’s output weight depends only on the input and on the type of the state, where the type of a K -1 
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bit state is defined to be the number of leading 0’s in the shift-register contents defining the state. We 
shall see that ZRL codes include as special cases orthogonal convolutional codes, the recent 
"superorthogonal codes" of Viterbi and many others. We will find that for any ZRL code, it is possible to 
compute the free distance by inspection, and to write down the complete transfer function T(D,IL,L), 
explicitly. In many cases, ZRL codes achieve the largest possible, or nearly the largest possible, free 
distance for their parameters. Furthermore, the ideas we have developed to analyze these specialized 
codes may well extend to a much larger class. 
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